cm0002@lemmy.world to Technology@lemmy.worldEnglish · 7 days agoOpen Source devs say AI crawlers dominate traffic, forcing blocks on entire countriesarstechnica.comexternal-linkmessage-square25fedilinkarrow-up1390arrow-down14
arrow-up1386arrow-down1external-linkOpen Source devs say AI crawlers dominate traffic, forcing blocks on entire countriesarstechnica.comcm0002@lemmy.world to Technology@lemmy.worldEnglish · 7 days agomessage-square25fedilink
minus-squaretaladar@sh.itjust.workslinkfedilinkEnglisharrow-up9·7 days agoThe tricky bit is recognizing that the requests are all from the same source. Often they use different IP addresses and to even classify requests at all you have to keep extra state around that you wouldn’t need without this anti-social behavior.
The tricky bit is recognizing that the requests are all from the same source. Often they use different IP addresses and to even classify requests at all you have to keep extra state around that you wouldn’t need without this anti-social behavior.