⚙️ Technical SEO

Fix the plumbing search engines rely on

Visitors rarely see technical problems, but those problems decide which URLs search engines crawl, which version they index and which language they show. Six of these tools write code you paste into your site, and two read what your server already sends.

Which tool for which job

• Robots.txt Generator: set disallowed paths, add your sitemap and optionally block AI crawlers such as GPTBot and ClaudeBot.
• Meta Robots Tag Generator: keep one page out of the index or out of snippets with noindex, nofollow or max-snippet.
• Canonical Tag Generator: point duplicate URLs to the version you want indexed.
• Hreflang Tag Generator: list every language or region version of a page plus an x-default, then paste the same set on every version.
• XML Sitemap Generator: turn a list of URLs into a sitemap file you can download.
• Htaccess Redirect Generator: write 301 or 302 rules, or force HTTPS, www or trailing slashes on Apache.
• Redirect Chain Checker: trace every hop in pasted curl output and get one .htaccess rule that skips the chain.
• HTTP Header Checker: grade X-Robots-Tag, caching, compression, HSTS, canonical and Vary headers.

A page blocked in robots.txt cannot be crawled, so search engines never see a noindex tag on it. If you want a page out of the index, let it be crawled and use noindex. The sitemap generator leaves lastmod out by default, and any date you pick applies to every URL, so add one only when it is the real date those pages changed. For the two checkers, pasted curl output gives the full result and stays in your browser; the quick live check sends your URL to a public proxy and shows only part of the picture.

For how crawl rules and wildcards interact, read The Complete robots.txt Reference: Precedence, Wildcards, AI Bots & Real-World Receipts.