How to Extract Sitemap URLs to CSV or JSON
A practical extraction workflow for turning nested XML sitemaps into a clean, deduplicated dataset you can audit or import.
Key takeaways
- Detect sitemap indexes before collecting URL entries.
- Normalize and deduplicate URLs without discarding meaningful parameters.
- Preserve lastmod and source sitemap fields for diagnostics.
- Export UTF-8 CSV with safe quoting or structured JSON for automation.
Define the extraction boundary
If you already have a urlset, extract its loc elements directly. If the document is a sitemap index, enqueue each child sitemap and repeat until the queue is empty or a safety limit is reached. Record which sitemap produced each URL; that provenance helps identify broken sections later.
Normalize without changing meaning
Resolve relative values against the response URL, accept only HTTP and HTTPS, remove fragments, and normalize obvious host casing. Do not blindly strip query strings: parameters may represent canonical pages in some applications. Let canonical checks decide whether a parameter belongs in the final audit set.
Choose useful output columns
A loc-only export is enough for a quick import, but an audit benefits from context. Keep the original sitemap, lastmod, changefreq, priority, extraction timestamp, and later validation results in separate columns.
- url
- source_sitemap
- lastmod
- http_status
- canonical_url
- indexability
Export JSON for applications
JSON preserves null values and nested metadata without CSV parsing rules. It is a good fit for APIs, queues, databases, and repeatable pipelines.
{
"url": "https://example.com/guides/xml-sitemaps",
"sourceSitemap": "https://example.com/sitemap-guides.xml",
"lastmod": "2026-07-18"
}Export CSV for spreadsheets and crawlers
Write UTF-8, quote fields that contain commas or line breaks, and neutralize cells that begin with spreadsheet formula characters when exporting untrusted values. Stream large exports rather than building the whole file in memory.
Once exported, compare the sitemap set with analytics landing pages, internal crawl URLs, and the CMS inventory. The differences reveal orphan pages, stale URLs, and pages that are internally linked but missing from the sitemap.
Parse a sitemap into a clean URL inventory.
Open the free toolFrequently asked questions
Can a sitemap index be exported directly?
It lists child sitemap files rather than page URLs. Traverse those child files first, then export their loc entries.
Should duplicate URLs be removed?
Yes for the final URL set, but retain source-sitemap provenance so you can detect duplicates across content sections.
Is CSV or JSON better?
CSV is convenient for spreadsheets and crawler imports; JSON is safer for structured application workflows. The source data can support both.
Primary references
See our technical methodology and editorial policy.