Review a small linked set of pages from one public website. Choose a limit of one to ten pages to inspect returned source HTML for missing titles, missing descriptions, duplicate titles, response issues, and detected noindex directives.
How to Use Website Crawl Report
- Enter a public starting URL without query parameters.
- Choose a maximum page count between one and ten.
- Run the crawl and review its page and time coverage statement.
- Investigate specific findings, edit the relevant pages, and recheck the changed URLs.
Use a small crawl to find repeated page issues
The crawler begins at the supplied URL, reads same-origin links from fetched HTML, and respects the fetched robots rules. It skips query and fragment variants during discovery and avoids selected resource-file extensions. Requests share a 40-second budget, so the report can be partial even below the page limit.
Example: review a small service website
Start at the homepage and check several linked service pages. If multiple pages return the same title, investigate whether a template is using a default instead of page-specific metadata. Review missing descriptions separately from HTTP or noindex findings; they do not all mean the same thing.
Read the coverage statement first
The tool reports how many pages were requested and whether the discovered queue was exhausted or a limit was reached. It does not render JavaScript, crawl every website page, or calculate an authority or ranking score. Robots unavailability or restricted access can stop a crawl instead of producing unsupported conclusions.
Use the On-Page SEO Checker for a detailed look at one finding. Use the Title and Description Checker while editing replacements and the SEO Checklist Builder to record the review workflow.
Key Features
- Bounded public website checks
- Example inputs
- Copy and download results
- No paid API or signup
Common Use Cases
- Website publishing
- Content review
- SEO maintenance
Privacy
The entered public URL is sent to our server for a bounded website check. Page content is processed for this request and is not saved by this tool.
Limitations
Respects fetched robots rules, skips query/fragment URLs, and checks source HTML. Maximum ten pages and 40 seconds per run; results may be partial.