Crawl & Sitemap Manager
Generate robots.txt, sitemap inventory, canonical host rules, and hreflang output in one workspace.
Loading tool…
浏览器内运行这个工具能做什么
Audit crawl policy, normalize page inventory, and prepare copy-ready robots, sitemap, and hreflang outputs from a single browser-based control surface.
Use it when you need to prepare robots rules, clean a sitemap inventory, and keep locale alternates aligned before release.
这个工具可以完成什么
Normalize page lists, blocked paths, sitemap candidates, and canonical host rules before a launch or domain migration.
Draft robots rules and compare them with the pages you still expect search engines to discover and index.
Keep sitemap inventory, hreflang output, and canonical host choices aligned when multiple locales or sections are involved.
依赖结果前需要检查什么
- Performance and maximum practical input size depend on browser memory, device speed, and the structure of the input.
- Review the generated result before replacing or publishing an original file.
如何使用
- 1
Enter the site URL, sitemap URL, canonical host, and locale setup.
- 2
Paste the current page inventory and blocked paths.
- 3
Review governance issues, then copy the robots, sitemap, hreflang, and summary outputs.
使用场景
Normalize page lists, blocked paths, sitemap candidates, and canonical host rules before a launch or domain migration.
Draft robots rules and compare them with the pages you still expect search engines to discover and index.
Keep sitemap inventory, hreflang output, and canonical host choices aligned when multiple locales or sections are involved.
使用技巧
Avoid putting redirected, blocked, duplicate, parameterized, or noindex URLs in the sitemap inventory you hand off.
If robots.txt disallows a path, do not also treat that path as a crawl-priority URL unless you have a specific migration reason.
常见问题
Does it crawl my live site automatically?
No. V1 is a deterministic governance workspace built from the inventory and rules you provide locally.
Is this replacing the focused robots, sitemap, canonical, and hreflang tools?
No. It becomes the main site-governance entry point, while the focused tools remain available for narrower tasks.
What inventory format works best?
Use one absolute URL per line for page inventory and blocked paths. Clean lists make canonical host checks, sitemap output, and robots rules easier to review.
下一步常用工具
如果还需要校验、转换或复用结果，可以继续打开相近的浏览器本地工具。