Create accountLogin

Recent searches

No recent searches

Search options

Only available when logged in.
openbiblio.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
Der Einstieg in das Fediverse für Bibliotheksmenschen

Administered by:

Server stats:

591
active users

openbiblio.social: AboutProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.3.8

Public
Adrian

It's good to see reporting on energy consumption for in-house data centers and third-party data centers going way up during the AI hype.

I wonder how much energy consumption went up because AI bots are massively hitting all the servers that are running our web. This definitely takes a lot of energy and time - from both machines and humans. Has anybody tried to estimate those numbers?

Public
Adrian

In my team, we had to step up our game this year having repeatedly experienced CPU overload on our servers due to massive bot scraping. This not only put quite some stress on the machines but also on people. I assume this happened at most places that are running their own data centers...

Sebastian Meyer

@acka47 Are you willing to share (at least part of) your config? I know of several libraries with similar problems, which are still losing the fight against .

Sep 16, 2024, 08:35 PM·Public
0boosts·1favorite
Quiet public
Adrian

@meyse It would be nice to get more people connected re. strategies against and learn from each other. We will soon start by addressing this at a hbz-wide level as teams are currently taking care of their services separately. However, I guess, we'd be happy to be part of a broader exchange about the topic. What do you think, @phu @dr0i ?

Vielleicht erstmal ein metadaten.community-Thema starten (auch wenn das Problem nicht nur Metadaten-Dienste betrifft)?

Quiet public *
dr0i

@acka47 @meyse @phu It's not just about so called : Today @phu identified massive nonesense queries done by user agent "facebookexternalhit/1.1" - a which even doesn't care about `robots.txt`. I am pro sharing configs and experience re crawling .

Quiet public
Jaoh91

@dr0i @acka47 @meyse @phu ist there some Kind of Platform to share Intel one User-Agents and IP Adresses ? This should also Bring some Kind of Automation to Update firewalls and configs/ Robots.txt

Quiet public
Sebastian Meyer

@Jaoh91 @dr0i @acka47 @phu I don't know. Searching for something like "ai bot blacklist" returns some results, but I haven't checked them in detail, yet.

ExploreLive feeds
About