# Web Scraper - Detailed Guide

> This file contains the detailed procedure and reference material extracted from `SKILL.md` for focused loading. The root skill defines activation, examples, safety constraints, and limitations.

## Overview

Web scraping inteligente multi-estrategia. Extrai dados estruturados de paginas web (tabelas, listas, precos). Paginacao, monitoramento e export CSV/JSON.


## How It Works

Execute phases in strict order. Each phase feeds the next.

```
1. CLARIFY  ->  2. RECON  ->  3. STRATEGY  ->  4. EXTRACT  ->  5. TRANSFORM  ->  6. VALIDATE  ->  7. FORMAT
```

Never skip Phase 1 or Phase 2. They prevent wasted effort and failed extractions.

**Fast path**: If user provides URL + clear data target + the request is simple
(single page, one data type), compress Phases 1-3 into a single action:
fetch, classify, and extract in one WebFetch call. Still validate and format.

---


## Escalation Consent

- Auto-escalation from WebFetch to Browser automation is allowed only for URLs
  the user explicitly provided.
- Discovery and ordinary pagination within an authorized research/extraction request can proceed after destination validation. Clarify only materially new scope or consequential actions.

---


## Capabilities

- **Multi-strategy**: WebFetch (static), Browser automation (JS-rendered), Bash/curl (APIs), WebSearch (discovery)
- **Extraction modes**: table, list, article, product, contact, FAQ, pricing, events, jobs, custom
- **Output formats**: Markdown tables (default), JSON, CSV
- **Pagination**: auto-detect and follow (page numbers, infinite scroll, load-more)
- **Multi-URL**: extract same structure across sources with comparison and diff
- **Validation**: descriptive coverage ratings (HIGH/MEDIUM/LOW), not calibrated probabilities
- **Auto-escalation**: WebFetch fails silently -> automatic Browser fallback
  (user-provided URLs only; see Escalation Consent)
- **Data transforms**: cleaning, normalization, deduplication, enrichment
- **Differential mode**: detect changes between scraping runs


## Web Scraper

Multi-strategy web data extraction with intelligent approach selection,
automatic fallback escalation, data transformation, and structured output.


## Phase 1: Clarify

Establish extraction parameters before touching any URL.


## Required Parameters

| Parameter     | Resolve                              | Default        |
|:--------------|:-------------------------------------|:---------------|
| Target URL(s) | Which page(s) to scrape?             | *(required)*   |
| Data Target   | What specific data to extract?       | *(required)*   |
| Output Format | Markdown table, JSON, CSV, or text?  | Markdown table |
| Scope         | Single page, paginated, or multi-URL?| Single page    |


## Optional Parameters

| Parameter     | Resolve                                | Default      |
|:--------------|:---------------------------------------|:-------------|
| Pagination    | Follow pagination? Max pages?          | No, 1 page   |
| Max Items     | Maximum number of items to collect?    | 500 per run; narrow or extend explicitly |
| Filters       | Data to exclude or include?            | None         |
| Sort Order    | How to sort results?                   | Source order  |
| Save Path     | Save to file? Which path?              | Display only |
| Language      | Respond in which language?             | User's lang  |
| Diff Mode     | Compare with previous run?             | No           |


## Clarification Rules

- If user provides a URL and clear data target, proceed directly to Phase 2.
  Do NOT ask unnecessary questions.
- If request is ambiguous (e.g. "scrape this site"), ask ONLY:
  "What specific data do you want me to extract from this page?"
- Default to Markdown table output. Mention alternatives only if relevant.
- Accept requests in any language. Always respond in the user's language.
- If user says "everything" or "all data", perform recon first, then present
  what's available and let user choose.


## Discovery Mode

When user has a topic but no specific URL:
1. Use WebSearch to find the most relevant pages
2. Present top 3-5 URLs with descriptions
3. Select relevant sources within the requested research scope; clarify only a material ambiguity
4. Proceed to Phase 2 with selected URL(s)

Example: "find and extract pricing data for CRM tools"
-> WebSearch("CRM tools pricing comparison 2026")
-> Select relevant public results within scope -> Extract

---


## Phase 2: Reconnaissance

Analyze the target page before extraction.


## Step 2.1: Initial Fetch

Use WebFetch to retrieve and analyze the page structure:

```
WebFetch(
  url = TARGET_URL,
  prompt = "Analyze this page structure and report:
    1. Page type: article, product listing, search results, data table,
       directory, dashboard, API docs, FAQ, pricing page, job board, events, or other
    2. Main content structure: tables, ordered/unordered lists, card grid, free-form text,
       accordion/collapsible sections, tabs
    3. Approximate number of distinct data items visible
    4. JavaScript rendering indicators: empty containers, loading spinners,
       SPA framework markers (React root, Vue app, Angular), minimal HTML with heavy JS
    5. Pagination: next/prev links, page numbers, load-more buttons,
       infinite scroll indicators, total results count
    6. Data density: how much structured, extractable data exists
    7. List the main data fields/columns available for extraction
    8. Embedded structured data: JSON-LD, microdata, OpenGraph tags
    9. Available download links: CSV, Excel, PDF, API endpoints"
)
```


## Step 2.2: Evaluate Fetch Quality

| Signal                                      | Interpretation                    | Action                    |
|:--------------------------------------------|:----------------------------------|:--------------------------|
| Rich content with data clearly visible      | Static page                       | Strategy A (WebFetch)     |
| Empty containers, "loading...", minimal text | JS-rendered                       | Strategy B (Browser)      |
| Login wall, CAPTCHA, 403/401 response       | Blocked                           | Report to user            |
| Content present but poorly structured       | Needs precision                   | Strategy B (Browser)      |
| JSON or XML response body                   | API endpoint                      | Strategy C (Bash/curl)    |
| Download links for CSV/Excel available      | Direct data file                  | Strategy C (download)     |


## Step 2.3: Content Classification

Classify into an extraction mode:

| Mode       | Indicators                                 | Examples                          |
|:-----------|:-------------------------------------------|:----------------------------------|
| `table`    | HTML `<table>`, grid layout with headers   | Price comparison, statistics, specs|
| `list`     | Repeated similar elements, card grids      | Search results, product listings  |
| `article`  | Long-form text with headings/paragraphs    | Blog post, news article, docs     |
| `product`  | Product name, price, specs, images, rating | E-commerce product page           |
| `contact`  | Names, emails, phones, addresses, roles    | Team page, staff directory        |
| `faq`      | Question-answer pairs, accordions          | FAQ page, help center             |
| `pricing`  | Plan names, prices, features, tiers        | SaaS pricing page                 |
| `events`   | Dates, locations, titles, descriptions     | Event listings, conferences       |
| `jobs`     | Titles, companies, locations, salaries     | Job boards, career pages          |
| `custom`   | User specified CSS selectors or fields     | Anything not matching above       |

Record: **page type**, **extraction mode**, **JS rendering needed (yes/no)**,
**available fields**, **structured data present (JSON-LD etc.)**.

If user asked for "everything", present the available fields and let them choose.

---


## Phase 3: Strategy Selection

Choose the extraction approach based on recon results.


## Decision Tree

```
Structured data (JSON-LD, microdata) has what we need?
 |
 +-- YES --> STRATEGY E: Extract structured data directly
 |
 +-- NO: Content fully visible in WebFetch?
      |
      +-- YES: Need precise element targeting?
      |    |
      |    +-- NO  --> STRATEGY A: WebFetch + AI extraction
      |    +-- YES --> STRATEGY B: Browser automation
      |
      +-- NO: JavaScript rendering detected?
           |
           +-- YES --> STRATEGY B: Browser automation
           +-- NO:  API/JSON/XML endpoint or download link?
                |
                +-- YES --> STRATEGY C: Bash (curl + jq)
                +-- NO  --> Report access issue to user
```


## Strategy A: Webfetch With Ai Extraction

**Best for**: Static pages, articles, simple tables, well-structured HTML.

Use WebFetch with a targeted extraction prompt tailored to the mode:

```
WebFetch(
  url = URL,
  prompt = "Extract [DATA_TARGET] from this page.
    Return ONLY the extracted data as [FORMAT] with these columns/fields: [FIELDS].
    Rules:
    - If a value is missing or unclear, use 'N/A'
    - Do not include navigation, ads, footers, or unrelated content
    - Preserve original values exactly (numbers, currencies, dates)
    - Include ALL matching items, not just the first few
    - For each item, also extract the URL/link if available"
)
```

**Auto-escalation**: If WebFetch returns suspiciously few items (less than
50% of expected from recon), or mostly empty fields, escalate to Strategy B.
Escalate automatically only for URLs the user explicitly provided; for
discovered URLs within scope, validate destinations and proceed (see Escalation Consent). Log the escalation in notes.


## Strategy B: Browser Automation

**Best for**: JS-rendered pages, SPAs, interactive content, lazy-loaded data.

Sequence:
These tool names are host-specific examples, not portable installed APIs. Discover the available connector/browser interface first and follow its documented restrictions.

1. Get tab context: `tabs_context_mcp(createIfEmpty=true)` -> get tabId
2. Navigate to URL: `navigate(url=TARGET_URL, tabId=TAB)`
3. Wait for content to load: `computer(action="wait", duration=3, tabId=TAB)`
4. Check for cookie/consent banners: `find(query="cookie consent or accept button", tabId=TAB)`
   - If found, dismiss it (prefer privacy-preserving option)
5. Read page structure: `read_page(tabId=TAB)` or `get_page_text(tabId=TAB)`
6. Locate target elements: `find(query="[DESCRIPTION]", tabId=TAB)`
7. Extract with JavaScript for precise data via `javascript_tool`

```javascript
// Table extraction
const rows = document.querySelectorAll('TABLE_SELECTOR tr');
const data = Array.from(rows).map(row => {
  const cells = row.querySelectorAll('td, th');
  return Array.from(cells).map(c => c.textContent.trim());
});
JSON.stringify(data);
```

```javascript
// List/card extraction
const items = document.querySelectorAll('ITEM_SELECTOR');
const data = Array.from(items).map(item => ({
  field1: item.querySelector('FIELD1_SELECTOR')?.textContent?.trim() || null,
  field2: item.querySelector('FIELD2_SELECTOR')?.textContent?.trim() || null,
  link: item.querySelector('a')?.href || null,
}));
JSON.stringify(data);
```

8. For lazy-loaded content, scroll and re-extract:
   `computer(action="scroll", scroll_direction="down", tabId=TAB)`
   then `computer(action="wait", duration=2, tabId=TAB)`


## Strategy C: Bash (Curl + Jq)

These are sketches. Bound response bytes and validate status/content type before parsing. Review redirects as new destinations; never forward credentials across origins. Use a new output path and refuse overwrites. Only fetch public HTTP(S) endpoints within scope; block loopback, private/link-local addresses, cloud metadata and DNS rebinding at every redirect. Do not interpolate page-derived URLs into shell commands; pass a validated URL as an argument. XML parsing requires a bounded parser configuration with entity/DTD expansion disabled.

**Best for**: REST APIs, JSON endpoints, XML feeds, CSV/Excel downloads.

```bash

## Json Api

curl --fail --silent --show-error --max-time 30 --proto "=https" "API_URL" | jq '[.items[] | {field1: .key1, field2: .key2}]'

## Csv Download

## Confirm The Exact Output Path With The User Before Downloading

curl --fail --silent --show-error --location \
  --max-time 30 --max-filesize 10485760 --proto "=https" --proto-redir "=https" \
  --output "<CONFIRMED_OUTPUT_PATH>/scraped_data.csv" -- "CSV_URL"

## Xml Parsing

curl --fail --silent --show-error --max-time 30 --proto "=https" "XML_URL" | python3 -c "
import xml.etree.ElementTree as ET, json, sys
tree = ET.fromstring(sys.stdin.read(10485761))  # Enforce a byte limit before parsing; reject oversize responses.

## ... Parse And Output Json

"
```


## Strategy D: Hybrid

When a single strategy is insufficient, combine:
1. WebSearch to discover relevant URLs
2. WebFetch for initial content assessment
3. Browser automation for JS-heavy sections
4. Bash for post-processing (jq, python for data cleaning)


## Strategy E: Structured Data Extraction

When JSON-LD, microdata, or OpenGraph is present:
1. Use Browser `javascript_tool` to extract structured data:
```javascript
const scripts = document.querySelectorAll('script[type="application/ld+json"]');
const data = Array.from(scripts).map(s => {
  try { return JSON.parse(s.textContent); } catch { return null; }
}).filter(Boolean);
JSON.stringify(data);
```
2. This often provides cleaner, more reliable data than DOM scraping
3. Fall back to DOM extraction only for fields not in structured data


## Pagination Handling

When pagination is detected and user wants multiple pages:

**Page-number pagination (any strategy):**
1. Extract data from current page
2. Identify URL pattern (e.g. `?page=N`, `/page/N`, `&offset=N`)
3. Iterate through pages up to user's max (default: 5 pages)
4. Show progress: "Extracting page 2/5..."
5. Concatenate all results, deduplicate if needed

**Infinite scroll (Browser only):**
1. Extract currently visible data
2. Record item count
3. Scroll down: `computer(action="scroll", scroll_direction="down", tabId=TAB)`
4. Wait: `computer(action="wait", duration=2, tabId=TAB)`
5. Extract newly loaded data
6. Compare count - if no new items after 2 scrolls, stop
7. Repeat until no new content or max iterations (default: 5)

**"Load More" button (Browser only):**
1. Extract currently visible data
2. Find button: `find(query="load more button", tabId=TAB)`
3. Click it: `computer(action="left_click", ref=REF, tabId=TAB)`
4. Wait and extract new content
5. Repeat until button disappears or max iterations reached

---


## Phase 4: Extract

Execute the selected strategy using mode-specific patterns.
See [references/extraction-patterns.md](../references/extraction-patterns.md)
for CSS selectors and JavaScript snippets.


## Table Mode

WebFetch prompt:
```
"Extract ALL rows from the table(s) on this page.
Return as a markdown table with exact column headers.
Include every row - do not truncate or summarize.
Preserve numeric precision, currencies, and units."
```


## List Mode

WebFetch prompt:
```
"Extract each [ITEM_TYPE] from this page.
For each item, extract: [FIELD_LIST].
Return as a JSON array of objects with these keys: [KEY_LIST].
Include ALL items, not just the first few. Include link/URL for each item if available."
```


## Article Mode

WebFetch prompt:
```
"Extract article metadata:
- title, author, date, tags/categories, word count estimate
- Key factual data points, statistics, and named entities
Return as structured markdown. Summarize the content; do not reproduce full text."
```


## Product Mode

WebFetch prompt:
```
"Extract product data with these exact fields:
- name, brand, price, currency, originalPrice (if discounted),
  availability, description (first 200 chars), rating, reviewCount,
  specifications (as key-value pairs), productUrl, imageUrl
Return as JSON. Use null for missing fields."
```

Also check for JSON-LD `Product` schema (Strategy E) first.


## Contact Mode

WebFetch prompt:
```
"Extract contact information for each person/entity:
- name, title, role, email, phone, address, organization, website, linkedinUrl
Return as a markdown table. Only extract real contacts visible on the page."
```


## Faq Mode

WebFetch prompt:
```
"Extract all question-answer pairs from this page.
For each FAQ item extract:
- question: the exact question text
- answer: the answer text (first 300 chars if long)
- category: the section/category if grouped
Return as a JSON array of objects."
```


## Pricing Mode

WebFetch prompt:
```
"Extract all pricing plans/tiers from this page.
For each plan extract:
- planName, monthlyPrice, annualPrice, currency
- features (array of included features)
- limitations (array of limits or excluded features)
- ctaText (call-to-action button text)
- highlighted (true if marked as recommended/popular)
Return as JSON. Use null for missing fields."
```


## Events Mode

WebFetch prompt:
```
"Extract all events/sessions from this page.
For each event extract:
- title, date, time, endTime, location, description (first 200 chars)
- speakers (array of names), category, registrationUrl
Return as JSON. Use null for missing fields."
```


## Jobs Mode

WebFetch prompt:
```
"Extract all job listings from this page.
For each job extract:
- title, company, location, salary, salaryRange, type (full-time/part-time/contract)
- postedDate, description (first 200 chars), applyUrl, tags
Return as JSON. Use null for missing fields."
```


## Custom Mode

When user provides specific selectors or field descriptions:
- Use Browser automation with `javascript_tool` and user's CSS selectors
- Or use WebFetch with a prompt built from user's field descriptions
- Use the already specified schema; clarify only if different sources require a material unresolved choice


## Multi-Url Extraction

When extracting from multiple URLs:
1. Extract from the **first URL** to establish the data schema
2. Validate the first results against the requested schema before continuing
3. Extract from remaining URLs using the same schema
4. Add a `source` column/field to every record with the origin URL
5. Combine all results into a single output
6. Show progress: "Extracting 3/7 URLs..."

---


## Phase 5: Transform

Clean, normalize, and enrich extracted data before validation.
See [references/data-transforms.md](../references/data-transforms.md) for patterns.


## Optional transforms (preserve raw values)

| Transform              | Action                                               |
|:-----------------------|:-----------------------------------------------------|
| Whitespace cleanup     | Trim, collapse multiple spaces, remove `\n` in cells |
| HTML entity decode     | `&amp;` -> `&`, `&lt;` -> `<`, `&#39;` -> `'`       |
| Unicode normalization  | NFKC only if safe for this field; retain raw text          |
| Empty string to null   | `""` -> `null` (for JSON), `""` -> `N/A` (for tables)|


## Conditional Transforms (Apply When Relevant)

| Transform             | When                         | Action                                  |
|:----------------------|:-----------------------------|:----------------------------------------|
| Price normalization   | Product/pricing modes        | Extract numeric value + currency symbol |
| Date normalization    | Any dates found              | Normalize to ISO-8601 (YYYY-MM-DD)      |
| URL resolution        | Relative URLs extracted      | Convert to absolute URLs                |
| Phone normalization   | Contact mode                 | Standardize to E.164 format if possible |
| Deduplication         | Multi-page or multi-URL      | Remove exact duplicate rows             |
| Sorting               | User requested or natural    | Sort by user-specified field            |


## Data Enrichment (Only When Useful)

| Enrichment             | When                         | Action                                |
|:-----------------------|:-----------------------------|:--------------------------------------|
| Currency conversion    | User asks for single currency| Note original + convert (approximate) |
| Domain extraction      | URLs in data                 | Add domain column from full URLs      |
| Word count             | Article mode                 | Count words in extracted text         |
| Relative dates         | Dates present                | Add "X days ago" column if useful     |


## Deduplication Strategy

When combining data from multiple pages or URLs:
1. Exact match: rows with identical values in all fields -> keep first
2. Near match: rows with same key fields (name+source) but different details
   -> preserve conflicting versions; merge only under a verified identity/precedence rule
3. Report: "Removed N duplicate rows" in delivery notes

---


## Phase 6: Validate

Verify extraction quality before delivering results.


## Validation Checks

| Check                | Action                                              |
|:---------------------|:----------------------------------------------------|
| Item count           | Compare extracted count to expected count from recon |
| Empty fields         | Count N/A or null values per field                   |
| Data type consistency| Numbers should be numeric, dates parseable           |
| Duplicates           | Flag exact duplicate rows (post-dedup)               |
| Encoding             | Check for HTML entities, garbled characters           |
| Completeness         | All user-requested fields present in output          |
| Truncation           | Verify data wasn't cut off (check last items)        |
| Outliers             | Flag values that seem anomalous (e.g. $0.00 price)  |


## Confidence Rating

These labels describe internal capture coverage only, not source truth. Report unknown expected totals as unknown; matching a page estimate cannot prove completeness.

| Rating     | Criteria                                                        |
|:-----------|:----------------------------------------------------------------|
| **HIGH**   | All fields populated, count matches expected, no anomalies      |
| **MEDIUM** | Minor gaps (<10% empty fields) or count slightly differs        |
| **LOW**    | Significant gaps (>10% empty), structural issues, partial data  |

Always report confidence with specifics:
> Confidence: **HIGH** - 47 items extracted, all 6 fields populated,
> matches expected count from page analysis.


## Auto-Recovery (Try Before Reporting Issues)

| Issue              | Auto-Recovery Action                                  |
|:-------------------|:------------------------------------------------------|
| Missing data       | Re-attempt with Browser if WebFetch was used          |
| Encoding problems  | Apply HTML entity decode + unicode normalization      |
| Incomplete results | Check for pagination or lazy-loading, fetch more      |
| Count mismatch     | Scroll/paginate to find remaining items               |
| All fields empty   | Page likely JS-rendered, switch to Browser strategy   |
| Partial fields     | Try JSON-LD extraction as supplement                  |

Log all recovery attempts in delivery notes.
Inform user of any irrecoverable gaps with specific details.

---


## Phase 7: Format And Deliver

Structure results according to user preference.
See [references/output-templates.md](../references/output-templates.md)
for complete formatting templates.


## Delivery Envelope

Include the relevant metadata in the requested format (a sidecar for CSV); do not prepend Markdown to machine-readable JSON/CSV:

```markdown

## Extraction Results

**Source:** [Page Title](http://example.com)
**Date:** YYYY-MM-DD HH:MM UTC
**Items:** N records (M fields each)
**Confidence:** HIGH | MEDIUM | LOW
**Strategy:** A (WebFetch) | B (Browser) | C (API) | E (Structured Data)
**Format:** Markdown Table | JSON | CSV

---

[DATA HERE]

---

**Notes:**
- [Any gaps, issues, or observations]
- [Transforms applied: deduplication, normalization, etc.]
- [Pages scraped if paginated: "Pages 1-5 of 12"]
- [Auto-escalation if it occurred: "Escalated from WebFetch to Browser"]
```


## Markdown Table Rules

- Left-align text columns (`:---`), right-align numbers (`---:`)
- Consistent column widths for readability
- Include summary row for numeric data when useful (totals, averages)
- Maximum 10 columns per table; split wider data into multiple tables
  or suggest JSON format
- Truncate long cell values to 60 chars with `...` indicator
- Use `N/A` for missing values, never leave cells empty
- For multi-page results, show combined table (not per-page)


## Json Rules

- Use camelCase for keys (e.g. `productName`, `unitPrice`)
- Wrap in metadata envelope:
  ```json
  {
    "metadata": {
      "source": "URL",
      "title": "Page Title",
      "extractedAt": "ISO-8601",
      "itemCount": 47,
      "fieldCount": 6,
      "confidence": "HIGH",
      "strategy": "A",
      "transforms": ["deduplication", "priceNormalization"],
      "notes": []
    },
    "data": [ ... ]
  }
  ```
- Pretty-print with 2-space indentation
- Numbers as numbers (not strings), booleans as booleans
- null for missing values (not empty strings)


## Csv Rules

- First row is always headers
- Quote any field containing commas, quotes, or newlines
- UTF-8 encoding with BOM for Excel compatibility
- Use `,` as delimiter (standard)
- Save metadata in a sidecar JSON/Markdown file; comment lines are not standard CSV.


## File Output

When user requests file save:
- Markdown: `.md` extension
- JSON: `.json` extension
- CSV: `.csv` extension
- Use the user-specified path or a suitable new local output; clarify only destructive overwrites or ambiguous destinations.
- Report full file path and item count after saving


## Multi-Url Comparison Format

When comparing data across multiple sources:
- Add `Source` as the first column/field
- Use short identifiers for sources (domain name or user label)
- Group by source or interleave based on user preference
- Highlight differences if user asks for comparison
- Include summary: "Best price: $X at store-b.com"


## Differential Output

When user requests change detection (diff mode):
- Compare current extraction with previous run
- Mark new items with `[NEW]`
- Mark removed items with `[REMOVED]`
- Mark changed values with `[WAS: old_value]`
- Include summary: "Changes since last run: +5 new, -2 removed, 3 modified"

---


## Rate Limiting

- Maximum 1 request per 2 seconds for sequential page fetches
- For multi-URL jobs, process sequentially with pauses
- If a site returns 429 (Too Many Requests), stop and report to user


## Access Respect

- If a page blocks access (403, CAPTCHA, login wall), report to user
- Do NOT attempt to bypass bot detection, CAPTCHAs, or access controls
- Do NOT scrape behind authentication unless user explicitly provides access
- Respect robots.txt directives when known


## Copyright

- Do NOT reproduce large blocks of copyrighted article text
- For articles: extract factual data, statistics, and structured info;
  summarize narrative content
- Always include source attribution (http://example.com) in output


## Data Scope

- Extract ONLY what the user explicitly requested
- Warn user before collecting potentially sensitive data at scale
  (emails, phone numbers, personal information)
- Store/export only within the authorized scope. The selected fetch/browser/AI provider may process page data; do not claim the workflow has no third-party processing.


## Failure Protocol

When extraction fails or is blocked:
1. Explain the specific reason (JS rendering, bot detection, login, etc.)
2. Suggest alternatives (different URL, API if available, manual approach)
3. Never retry aggressively or escalate access attempts

---


## Quick Reference: Mode Cheat Sheet

| User Says...                         | Mode      | Strategy  | Output Default   |
|:-------------------------------------|:----------|:----------|:-----------------|
| "extract the table"                  | table     | A or B    | Markdown table   |
| "get all products/prices"            | product   | E then A  | Markdown table   |
| "scrape the listings"                | list      | A or B    | Markdown table   |
| "extract contact info / team page"   | contact   | A         | Markdown table   |
| "get the article data"               | article   | A         | Markdown text    |
| "extract the FAQ"                    | faq       | A or B    | JSON             |
| "get pricing plans"                  | pricing   | A or B    | Markdown table   |
| "scrape job listings"                | jobs      | A or B    | Markdown table   |
| "get event schedule"                 | events    | A or B    | Markdown table   |
| "find and extract [topic]"           | discovery | WebSearch | Markdown table   |
| "compare prices across sites"        | multi-URL | A or B    | Comparison table |
| "what changed since last time"       | diff      | any       | Diff format      |

---


## References

- **Extraction patterns**: [references/extraction-patterns.md](../references/extraction-patterns.md)
  CSS selectors, JavaScript snippets, JSON-LD parsing, domain tips.

- **Output templates**: [references/output-templates.md](../references/output-templates.md)
  Markdown, JSON, CSV templates with complete examples.

- **Data transforms**: [references/data-transforms.md](../references/data-transforms.md)
  Cleaning, normalization, deduplication, enrichment patterns.


## Best Practices

- Provide clear, specific context about your project and requirements
- Review all suggestions before applying them to production code
- Combine with other complementary skills for comprehensive analysis


## Common Pitfalls

- Using this skill for tasks outside its domain expertise
- Applying recommendations without understanding your specific context
- Not providing enough project context for accurate analysis