Anubis (software)
Anubis | |
---|---|
![]() The page shown by Anubis during its challenge. | |
Original author(s) | Xe Iaso |
Developer(s) | Techaro[1] |
Initial release | January 19, 2025[1][2] |
Stable release | 1.22.0[3] ![]() |
Repository | github |
Written in | Go, JavaScript[1] |
License | MIT License |
Website | anubis |
Anubis is a software program that adds a JavaScript-powered proof of work challenge to websites before users can access them, as a form of deterring the impacts of web scraping from bots. It has been adopted mainly by Git forges and free and open-source software projects.[4][5]
It was created by Xe Iaso in response to Amazon's web crawler overloading their Git server, as it did not respect the robots.txt exclusion protocol and would work around restrictions.[4][6]
Adoption
[edit]It is used by a number of projects, including:[7]
- The GNOME Project's GitLab instance[4]
- The Linux kernel mailing list archives[1] and Git server[8]
- sourceware.org[7]
- FFmpeg[1][6]
- FreeBSD
- FreeDesktop
- Wine[9]
- UNESCO[6]
- The Science Olympiad Student Center[9]
- Enlightenment[9]
- FreeCAD[10]
- Krantenbank Zeeland
- ScummVM[11]
Criticism
[edit]Tavis Ormandy criticized the project, citing the low financial cost for scraping companies to calculate the hashes.[12]
References
[edit]- ^ a b c d e Brockmeier, Joe (July 10, 2025). "Anubis sends AI scraperbots to a well-deserved fate". LWN.net.
- ^ Iaso, Xe (January 19, 2025). "Block AI scrapers with Anubis". xeiaso.net. Retrieved May 2, 2025.
- ^ "Release 1.22.0". September 6, 2025. Archived from the original on September 7, 2025. Retrieved September 7, 2025.
- ^ a b c Edwards, Benj (March 25, 2025). "Open source devs say AI crawlers dominate traffic, forcing blocks on entire countries". Ars Technica. Retrieved May 2, 2025.
- ^ Bort, Julie (March 27, 2025). "Open source devs are fighting AI crawlers with cleverness and vengeance". TechCrunch. Retrieved May 2, 2025.
- ^ a b c Maiberg, Emanuel (July 7, 2025). "The Open-Source Software Saving the Internet From AI Bot Scrapers". 404 Media. Retrieved July 14, 2025.
- ^ a b "List of known websites using Anubis". anubis.techaro.lol. Retrieved May 10, 2025.
- ^ "Anubis". Techaro. Retrieved July 28, 2025.
- ^ a b c Iaso, Xe (April 12, 2025). "Anubis works". xeiaso.net. Retrieved May 9, 2025.
- ^ "Forum and Wiki Performance; Anubis Deployed". FreeCAD News. April 30, 2025. Retrieved May 11, 2025.
- ^ "The Day Anubis Saved Our Websites From a DDoS Attack". fabulous.systems. May 1, 2025. Retrieved June 19, 2025.
- ^ Ormandy, Tavis. "Anubis". lock.cmpxchg8b.com. Retrieved September 3, 2025.