--- name: afrexai-technical-seo description: "Technical SEO Mastery" --- # Technical SEO Mastery > Complete technical SEO audit, fix, and monitoring system. From crawlability to Core Web Vitals to international SEO — everything search engines need to find, crawl, index, and rank your site. ## When to Use - "Technical SEO audit for my site" - "Why isn't Google indexing my pages?" - "Fix Core Web Vitals issues" - "Pre-migration SEO checklist" - "My site is slow / rankings dropped" - "Check robots.txt / sitemap / schema markup" - "International SEO / hreflang setup" --- ## Phase 1: Quick Health Check (5-Minute Triage) Before a full audit, run a rapid assessment. Fetch the site and check: ```yaml quick_health: url: "[domain]" checks: robots_txt: "[accessible / blocked / missing]" sitemap_xml: "[found / missing / errors]" https: "[yes / mixed content / no]" mobile_viewport: "[set / missing]" page_load: "[fast <2s / moderate 2-4s / slow >4s]" h1_present: "[yes / missing / multiple]" canonical: "[set / missing / self-referencing]" structured_data: "[present / missing / errors]" severity: "[healthy / needs work / critical]" priority_fix: "[top issue to address first]" ``` **Severity guide:** - 🟢 Healthy: 0-1 issues — minor optimizations only - 🟡 Needs work: 2-4 issues — schedule fixes this week - 🔴 Critical: 5+ issues or any blocking issue — fix immediately --- ## Phase 2: Crawlability Audit ### 2.1 Robots.txt Analysis Fetch `[domain]/robots.txt` and evaluate: ```yaml robots_txt_audit: exists: true/false valid_syntax: true/false issues: - type: "[blocked_important_page / missing_sitemap / wildcard_block / syntax_error]" detail: "[specific line or pattern]" severity: "critical/warning/info" fix: "[exact fix]" checks: - "Sitemap directive present" - "No accidental blocking of CSS/JS/images" - "No blocking of important page directories" - "Correct user-agent targeting (Googlebot, Bingbot, etc.)" - "No conflicting rules (allow + disallow same path)" - "Crawl-delay only if needed (slows indexing)" ``` **Common mistakes:** | Mistake | Impact | Fix | |---------|--------|-----| | `Disallow: /` blocking everything | No pages indexed | Remove or narrow scope | | Blocking CSS/JS | Poor rendering = ranking drop | `Allow: /assets/` | | No sitemap reference | Slower discovery | Add `Sitemap:` directive | | Multiple sitemaps not declared | Partial crawling | Declare all sitemaps | | Blocking search/filter pages poorly | Crawl waste | Use `Disallow: /*?` patterns | **Recommended template:** ``` User-agent: * Allow: / Disallow: /admin/ Disallow: /api/ Disallow: /checkout/ Disallow: /*?sort= Disallow: /*?filter= Sitemap: https://example.com/sitemap.xml Sitemap: https://example.com/sitemap-images.xml ``` ### 2.2 XML Sitemap Audit Fetch sitemap(s) and evaluate: ```yaml sitemap_audit: url: "[sitemap URL]" type: "[index / single / none]" url_count: 0 issues: - type: "[missing / too_large / includes_noindex / stale_lastmod / broken_urls]" severity: "critical/warning/info" fix: "[specific action]" quality_checks: - "All URLs return 200 (no 404s, 301s, or 5xx)" - "No noindex pages included" - "lastmod dates are accurate (not all same date)" - "Under 50,000 URLs per sitemap file" - "Under 50MB uncompressed per file" - "Sitemap index if >50K URLs" - "Matches canonical URLs (not alternate versions)" - "Images/video sitemaps if applicable" ``` **Sitemap best practices:** - One URL per `` entry — canonical version only - `lastmod` should reflect actual content change date - Priority and changefreq are largely ignored by Google — optional - Compress with gzip for large sitemaps - Submit in Google Search Console AND robots.txt ### 2.3 Crawl Budget Optimization ```yaml crawl_budget_analysis: total_pages: 0 indexable_pages: 0 crawl_waste_ratio: "[indexable / total — target >80%]" waste_sources: - source: "[faceted navigation / pagination / parameters / duplicate content / thin pages]" page_count: 0 action: "[noindex / canonical / robots block / parameter handling / consolidate]" optimization_priority: 1: "Remove/noindex thin and duplicate pages" 2: "Consolidate parameter variations with canonicals" 3: "Implement pagination best practices (rel=next or load-more)" 4: "Fix redirect chains (max 1 hop)" 5: "Eliminate orphan pages or link them into site structure" ``` **Crawl budget matters when:** Site has >10K pages OR crawl rate is notably low in Search Console. --- ## Phase 3: Indexability Audit ### 3.1 Index Status Check For each important page type, verify: ```yaml indexability_check: page_type: "[homepage / product / blog / category / landing]" sample_url: "[URL]" signals: meta_robots: "[index,follow / noindex / nofollow / none]" x_robots_tag: "[present / absent — check HTTP headers]" canonical: "[self / points to other URL / missing]" http_status: "[200 / 301 / 302 / 404 / 410 / 5xx]" in_sitemap: true/false internal_links_to: "[count of internal links pointing here]" robots_txt_allowed: true/false verdict: "[indexable / blocked / conflicting signals]" fix: "[action if not indexable]" ``` ### 3.2 Common Indexing Blockers | Blocker | Detection | Fix | Priority | |---------|-----------|-----|----------| | `noindex` meta tag | Check `` | Remove tag or move to correct pages | P0 | | `X-Robots-Tag: noindex` header | Check HTTP response headers | Remove header from server config | P0 | | Canonical pointing elsewhere | Check `` | Fix to self-reference or correct target | P0 | | Blocked in robots.txt | Cross-reference robots.txt | Update robots.txt rules | P0 | | Not in sitemap | Check sitemap inclusion | Add to sitemap | P1 | | No internal links (orphan) | Crawl internal link graph | Add contextual internal links | P1 | | Soft 404 (200 with no content) | Check page content | Return proper 404 or add content | P1 | | Duplicate content | Compare page similarity | Canonical or consolidate | P2 | | Thin content (<200 words, no value) | Word count + quality check | Expand or merge with related page | P2 | ### 3.3 Redirect Audit ```yaml redirect_audit: chains_found: 0 # A→B→C (should be A→C) loops_found: 0 # A→B→A (broken) temporary_redirects: 0 # 302s that should be 301s rules: - "Max 1 redirect hop (no chains)" - "Use 301 for permanent moves, 308 for POST-preserving" - "302 only for genuinely temporary redirects" - "Update internal links to point to final destination" - "Redirect HTTP → HTTPS at server level" - "Redirect www ↔ non-www consistently" ``` --- ## Phase 4: Core Web Vitals & Performance ### 4.1 Core Web Vitals Targets | Metric | Good | Needs Improvement | Poor | |--------|------|-------------------|------| | **LCP** (Largest Contentful Paint) | ≤2.5s | 2.5-4.0s | >4.0s | | **INP** (Interaction to Next Paint) | ≤200ms | 200-500ms | >500ms | | **CLS** (Cumulative Layout Shift) | ≤0.1 | 0.1-0.25 | >0.25 | ### 4.2 LCP Fix Playbook ```yaml lcp_diagnosis: current_lcp: "[Xs]" lcp_element: "[identify the largest element — usually hero image or heading]" fix_priority: 1: issue: "Slow server response (TTFB >800ms)" fixes: - "Enable server-side caching (Redis, Varnish)" - "Use CDN for static assets" - "Optimize database queries" - "Upgrade hosting if shared" 2: issue: "Render-blocking resources" fixes: - "Defer non-critical CSS: ``" - "Async/defer JavaScript: `