CLI
The package ships a urls binary. Install it globally or run it through npx.
pnpm add -g @agntn/urls
Discover
urls discover example.com -p wayback -n 20
One URL per line, nothing else, so it pipes straight into grep, sort -u or whatever you feed URLs to. A reader that quits early, like head, ends the command quietly. No EPIPE stack trace confetti.
| Flag | What it does |
|---|---|
-p, --provider | Source key, or all for every source |
-n, --limit | Stop after this many URLs per source |
-m, --match | Keep URLs containing one of these substrings |
-f, --filter | Drop URLs containing one of these substrings |
-us, --url-scope | Keep URLs matching a prefix or glob |
-uos, --url-out-scope | Drop URLs matching a prefix or glob |
--ext | Keep these path extensions |
--has-query | Keep URLs with real query keys |
--from, --to | Capture date window, 2019 or 2019-03-15 |
-ns, --no-scope | Turn off the default host check |
-j, --jsonl | One JSON record per line |
Lists are comma separated. Filters and scope explains what each filter keeps.
JSONL
urls discover example.com -p wayback -n 2 -j
{"url":"http://example.com:80/","input":"example.com","source":"wayback","firstSeen":"2002-01-20T14:25:10Z","lastSeen":"2002-01-20T14:25:10Z"}
{"url":"http://www.example.com:80/","input":"example.com","source":"wayback","firstSeen":"2002-03-28T01:28:21Z","lastSeen":"2002-03-28T01:28:21Z"}
Fields you don't have stay out of the line instead of showing up as null.
Every source at once
urls discover example.com -p all -n 20
Each source gets its own block with a count, and a failing source gets one line with its error. With -j you get the successful records only, each tagged with its source.
Without -p
The CLI picks a source the way discoverWithFallback does: configured credentials first, then AlienVault OTX, then the next one when access fails. Discovering URLs has the rules.
Providers
urls providers
Prints the registered sources with their endpoints and whether they need a key.
MCP server
urls mcp
Starts the MCP server on stdio. Agents shows the client config.
Exit codes
0 when the command finished, 1 on a bad limit or a failed discovery. A comparison with -p all exits 0 even when some sources failed, since their errors are part of the answer.