Providers
Arquivo.pt
The CDX index of the Portuguese web archive. No key. One NDJSON request with dates and a hard cap of 10 000 rows.
Source
Arquivo.pt
- provider: "arquivo"
- arquivo.pt
The Portuguese web archive. CDX with dates, and a hard cap per request.
- Format
- CDX NDJSON
- Paging
- one request, 10 000 rows
- Dates
- firstSeen · lastSeen
- Key
- none
Access
- Load
await create("arquivo") - CLI
urls discover example.com -p arquivo - Key
nothing, no key and no account - Endpoint
https://arquivo.pt/wayback/cdx
What it knows
Arquivo.pt archives the Portuguese web first, but don't let the name fool you: nuxt.com and example.com both come back with captures. Every row carries a timestamp, so firstSeen, lastSeen, from and to all work.
How it asks
One request to /wayback/cdx with matchType=domain, output=json and fields=url,timestamp, so subdomains come along. The answer is NDJSON, one capture per line, streamed and parsed as it arrives.
Traps
- Without a
limitthe stream answers 200 with an empty body and hangs, so the provider always sendslimit=10000. An answer that fills all 10 000 rows may have been cut by the archive, and the page says so withtruncated. output=txtis rejected, which is why this one speaks NDJSON while Wayback speaks text.- 10 000 rows are captures, not URLs. A popular page captured a thousand times eats a thousand rows and gives you one URL.