Robots.txt Generator
Build, validate, and download a working robots.txt in 60 seconds. Templates for WordPress, Shopify, Next.js.
Search and quickly navigate to tools.
Free, browser-based utilities covering the technical-SEO workload: crawl control, structured data, on-page meta inspection, sitemap generation, AI-crawler discovery, and domain intelligence. Pair them in the order suggested below for a complete pre-launch or audit pass.
Build, validate, and download a working robots.txt in 60 seconds. Templates for WordPress, Shopify, Next.js.
Inspect title, description, OG, Twitter Card, canonical, and robots tags for any URL in one scan.
Build JSON-LD for Article, FAQ, HowTo, Product, Local Business, Recipe, Event, Organization, and more.
Crawl your site and generate a sitemap.xml ready for Google Search Console submission.
Compose title, description, Open Graph, and Twitter Card markup with character-count guidance.
Publish an llms.txt manifest so AI crawlers can discover your site’s most authoritative content.
Run through a structured technical, on-page, and content checklist before any major release.
Check the registration date, age, registrar, and expiration of any domain via the public RDAP registry.
You do not need a $400/month platform to ship a technically correct site. The eight tools above cover the 90% of technical-SEO execution that most teams actually do — generating crawl rules, validating tags, producing structured data, and emitting sitemap and discovery manifests. Below is the order we recommend for a new site, a redesign, or a quarterly audit.
Use the Robots.txt Generator to produce a deliberate allow/disallow file. The default templates cover WordPress, Shopify, Next.js, and static-site setups. Critical mistakes here — accidentally blocking / or leaving staging rules in production — are the single most common cause of a sudden indexing collapse. Pair the file with an explicit reference to your sitemap URL at the bottom so Google discovers structured updates faster.
Use the Sitemap Generator to crawl your live site and produce a clean sitemap.xml. Modern Next.js, Astro, and SvelteKit sites typically ship one automatically, but the manual generator catches orphan pages and lets you exclude staging or admin routes that should not be indexed. Submit the result through Google Search Console → Sitemaps and watch the Coverage report over the following two weeks.
Run the Meta Tags Checker against your homepage, one product or article page, one category page, and a blog post. The checker reports title length, description character count, Open Graph and Twitter Card completeness, canonical correctness, and robots directives — the surface area that drives SERP rendering. Anything outside the 50-60 char title and 120-160 char description ranges leaks CTR on the search results page.
Use the Schema Markup Generator for FAQ, HowTo, and Product schemas — the three highest-ROI rich-result types for most sites. Validate every snippet with Google's Rich Results Test before shipping. Structured data is not a ranking factor on its own, but it unlocks visual SERP features (FAQ accordion, star ratings, image carousels) that materially increase CTR even at the same position.
Use the LLMs.txt Generator to produce a curated index of your most authoritative pages for AI crawlers. The format is new and adoption is uneven, but the cost of publishing is essentially zero and the upside is being correctly cited by ChatGPT, Perplexity, and Anthropic-family agents as they evolve their crawling.
Use the Domain Age Checker during competitive analysis or vendor due diligence. The age of a competitor's domain explains a lot about why an otherwise weaker site might still outrank you — it has had years to accumulate trust signals. The same lookup also helps triage suspicious or phishing domains during incident response.
Finally, walk through the SEO Checklist as a final gate before any production push. It covers the structural items the tools above each address individually, plus the softer signals (page speed, internal linking, image alt text, broken links) that compound over time.
Free tools cover execution: generating files, building markup, validating tags. Paid platforms (Ahrefs, Semrush, Moz, Sitebulb) cover intelligence: keyword research, backlink discovery, rank tracking, competitive gap analysis. Most teams need both, but the execution-side spend is genuinely zero — there is no quality reason to pay $30/month for a meta-tag checker.
If you are building or running a small site (under 1,000 indexed pages), the free stack on this hub plus Google Search Console is usually enough. The moment you start needing competitor backlink data, weekly rank tracking, or large-scale crawl audits is when a paid platform starts to pay for itself.