Crawl any public website and get a flat list of every internally-linked URL. Sortable, filterable, and one-click CSV export. Capped at 1000 URLs, depth 8. Built for SEO audits, content inventories, and migration prep.
1. Configure crawl
URL
Host
Scheme
Depth
What this tool does
Starting from your root URL, we do a breadth-first crawl: fetch the page, parse every <a href>, follow each internal link, and repeat. Each unique URL is added to the result, along with the host, scheme, and path depth.
Why path depth matters
Depth 0–2: homepage and top-level sections — these get crawled fastest by Google.
Depth 3–4: typical article and product pages — still well-indexed.
Depth 5+: deep pages that may take Google weeks to find. Sort by Depth desc to spot orphan content.
Use cases
Pre-migration audit: get a full URL list before swapping CMSes
SEO inventory: feed the CSV into your spreadsheet to map redirects
Sitemap recovery: when you've lost track of what pages exist on an old domain
Pre-CustomMind: see exactly which pages our chatbot crawler will index