Getting Started

Why would you want this?

Wayback remembers one URL, AlienVault OTX knows another, URLScan saw a third one last week. Each of them speaks its own dialect: CDX text, CDX as NDJSON, paged JSON, search cursors. Ask all of them yourself and you're writing a client per source for what sounded like one question.

@agntn/urls asks them for you. Same call, same options, same record back. It's a TypeScript take on the question ProjectDiscovery's urlfinder answers.

What "passive" means here

Requests go to the sources, never to the domain you're asking about. The site doesn't see a single hit from you. What you get back is what somebody else saw, sometimes years ago.

Install

Node.js 26 or newer.

pnpm add @agntn/urls

The npm release is still pending, so for now this works from a checkout.

First call from the CLI

urls discover example.com -p wayback -n 2

http://example.com:80/
http://www.example.com:80/

No API key needed. Even example.com has a past ;) The :80 is how Wayback wrote it down in 2002, and the library keeps the first spelling it saw instead of prettifying it.

Without -p the CLI picks a source for you. Discovering URLs explains which one and when it moves on.

First call from code

import { create } from "@agntn/urls";

const wayback = await create("wayback");
const hits = await wayback.discover("example.com", { limit: 20 });

for (const hit of hits) {
  console.log(hit.source, hit.url, hit.firstSeen);
}

create() is async because each source lives in its own module and loads on first use. Ask for Wayback and the others never leave the disk.

You get records, not a blob to parse again. Discovering URLs lists the fields.

What it doesn't do

No crawling, no page downloads, no check that a URL still answers. A hit means some source saw the URL once. Whether it's alive today is a different question, and a different tool.

Where next

• Discovering URLs: one source, the automatic pick, or all of them side by side.
• Filters and scope: keep what you came for.
• Agents: MCP, AI SDK, Pi and OMP.
• Providers: what each source knows and what it needs.