anubis is cancer

Ah, Anubis. The AI scraper blocker that used to have an AI generated anime girl as a mascot!

If you read the title, you know this will be a rant.

Old Browsers

Forget about using old browsers or any TUI based ones (links my beloved). You need modern JavaScript features.

Disabling JavaScript

Even if you don’t use an ancient browser, you may still be affected by the JavaScript usage.

I use NoScript, an incredible JavaScript blocker. Anubis uses Proof of work checks that are written in JavaScript to determine if a user is a scraper. This requires me to whitelist (or at least temporarily whitelist) EVERY SINGLE ANUBIS SITE.

BUT WHAT ABOUT THE SCRAPERS

I have 2 solutions.

1. Just don’t give a fuck. Not using any bot protection is fine for most people.

Or…

Actually good bot protection

Make a site that gets displayed that has a message like “To prevent abuse please press this button” and a button. This should all be wrapped in a form element.

Make a backend service that listens on an endpoint the form POSTs to. Whenever a POST request is received, make it set a cookie.

For every request the backend service should check if the cookie exists and if it does it should open the website.

I might make this eventually and publish it on git.forever2k38.net, be on the lookout!