ia_arc
Archiveby Internet Archive
ia_archiver
The Internet Archive's crawler for saving snapshots of web pages to the Wayback Machine (web.archive.org). Respects robots.txt.
Respects robots.txt
Yes
Can be blocked
Yes
Crawl-Delay support
No
Type
Archive
Purpose
Archiving web pages for the Wayback Machine
SEO Impact
Blocking ia_archiver prevents the Wayback Machine from archiving your site. This removes historical snapshots, which can matter for legal/compliance, reputation management, or preserving content history.
User-Agent String
ia_archiver
robots.txt Control
Add "User-agent: ia_archiver" with "Disallow: /" in robots.txt.
Block
User-agent: ia_archiver Disallow: /
Allow (default)
User-agent: ia_archiver Allow: /
Official Documentation
Test your robots.txt against ia_archiver
Check which paths are blocked or allowed for each user-agent