--- name: goofish-search-list description: "Scrapes second-hand item search results from Goofish (闲鱼/xianyu, goofish.com) — China's largest second-hand marketplace. Input: keyword, optional sort/filter params. Output: list of items with id, title, price, image, location, want-count per page (30 items/page). Use when user mentions goofish, 闲鱼, xianyu, 二手交易, second-hand marketplace China, 二手商品搜索, search used goods, scrape goofish listings, xianyu search results, collect second-hand prices, monitor used item prices, 闲鱼关键词搜索, 闲鱼数据采集, 批量抓取闲鱼, goofish scraper, goofish data, xianyu data extraction, 二手商品价格监控, used iPhone prices, 二手手机价格. Also applies to: price research on Chinese second-hand market, competitor product monitoring via used goods listings, inventory analysis." --- # Goofish (闲鱼) — Search Results List > keyword + optional filters → list of 30 second-hand item cards per page (id, title, price, image, location, want-count) ## Language All process output to user (progress updates, process notifications) follows the user's language. ## Objective Extract second-hand item listing cards from Goofish keyword search results, supporting sort options, price range filters, and publish-date filters, with page-by-page pagination. ## Prerequisites - Browser with an active Goofish session (logged-in account recommended for full results) - Target page is already open or will be opened: `https://www.goofish.com/search?q={keyword}` ## Pre-execution Checks ### 1. Tool Readiness If browser-act has been confirmed available in the current session → skip this step. Invoke `browser-act` via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry. ### 2. Login Verification If login status for Goofish has been confirmed in the current session → skip this step. Otherwise: open `https://www.goofish.com/` and observe the page: - User avatar or account entry exists → logged in, continue - Login/register prompt → not logged in; inform user that login may be required for full results; assist login if needed ## Capability Components > This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page. JS code is encapsulated in Python files under the `scripts/` directory, invoked via `eval "$(python scripts/xxx.py {params})"`. `$(...)` is bash syntax; it is recommended to use the bash tool for execution. ### Network Capture: trigger search and load results Search requests use a dynamic `sign` token computed client-side — they cannot be reconstructed directly. Navigate to the search URL to trigger the API automatically. 1. `navigate https://www.goofish.com/search?q={keyword}` 2. `wait stable` 3. Proceed to DOM extraction below Error handling: If the page shows a CAPTCHA slider ("Please slide to verify") instead of search results, the session has been rate-limited. Wait 5–10 minutes before retrying, or switch to a fresh browser session. ### DOM: search result item cards (data extraction) After navigating and waiting stable, extract all 30 item cards on the current page: `eval "$(python scripts/extract-search-items.py)"` Output example: ```json { "items": [ { "item_id": "1054668718340", // unique item ID "category_id": "126862528", // category ID "item_url": "https://www.goofish.com/item?id=1054668718340&categoryId=126862528", "title": "美版iPhone 14 国行256G 纯原 原版原漆", // full title text "image_url": "https://img.alicdn.com/bao/uploaded/...", // thumbnail URL "price": "1810", // numeric string, CNY, no ¥ sign "service_tag": "Apple/苹果256GB无任何维修", // condition/attribute tag or recency label, null if absent "price_desc": "2人想要", // want-count or price-drop info, null if absent "location": "广东" // seller's location province/city } ], "count": 30 } ``` ### DOM: apply sort and filter options (operation) Apply sort order, publish-date filter, or price range before extracting. Call before running `extract-search-items.py`. After calling, `wait stable` before extracting. `eval "$(python scripts/apply-search-filters.py --sort {sort} --publish-days {days} --price-min {min} --price-max {max})"` Parameters: - `--sort`: Sort option — `""` default (综合), `"reduce"` price-drop (新降价), `"create"` newest (新发布), `"price-asc"` price low-to-high, `"price-desc"` price high-to-low. Default: `""` - `--publish-days`: Filter by publish date — `""` all, `"1"` within 1 day, `"3"` within 3 days, `"7"` within 7 days, `"14"` within 14 days. Default: `""` - `--price-min`: Minimum price (CNY integer string, e.g., `"500"`). Requires `--price-max`. Default: `""` - `--price-max`: Maximum price (CNY integer string, e.g., `"3000"`). Requires `--price-min`. Default: `""` Output example: ```json { "ok": true, "applied": { "sort": "reduce:desc", "searchFilter": "publishDays:7;priceRange:500,3000;" } } ``` ### DOM: navigate to a specific page (operation) `eval "$(python scripts/goto-page.py {page_number})"` Parameters: - `page_number`: Target page number (integer, 1-based) Output example: ```json { "ok": true, "clicked_page": 2 } ``` After clicking, `wait stable` then re-run `extract-search-items.py` to get the new page's items. ## Enum Parameters [AI] sort options: `""` (综合/default), `"reduce"` (新降价), `"create"` (新发布/最新), `"price-asc"` (价格从低到高), `"price-desc"` (价格从高到低) [AI] publish-days filter: `""` (all), `"1"`, `"3"`, `"7"`, `"14"` ## Pagination **DOM Pagination**: Click the target page number button using `goto-page.py {page}`, then `wait stable`, then re-run `extract-search-items.py`. Page numbers appear in the pagination bar at the bottom of the search results. Termination: When `goto-page.py` returns `error: Page N not found` — no more pages available, or the target page exceeds the pagination range displayed (typically up to 25 pages / 750 items). ## Success Criteria `result count >= 1` and `item_id non-null rate = 100%` and `price non-null rate >= 80%` ## Known Limitations - 30 items per page (fixed by the site) - Maximum ~750 items accessible via pagination (25 pages × 30) - Seller username and user ID are not available in search cards — only seller location - Session rate limiting: accessing item detail pages rapidly after heavy search usage may trigger a CAPTCHA slider; mitigate by adding 1–2 second delays between page navigations - The `sign` token in search API requests is computed client-side; direct API replay without browser context is not supported — always trigger via page navigation ## Execution Efficiency - **Batch orchestration**: Write a bash script to loop through keywords serially within a single session; do not parallelize within one browser (prone to triggering anti-scraping). Add 1–2 second delays between page navigations. To increase throughput, open multiple stealth browser sessions and distribute keywords across them - **Test before batch execution**: After writing a batch script, first test with 1–2 keywords/pages to verify the script runs correctly; only then run the full batch - **Reduce redundant pre-operations**: Navigate once per keyword, apply all filters at once before extracting, rather than navigating multiple times - **Error resumption**: Save results keyword-by-keyword and page-by-page during batch processing; on failure, resume from the last saved position ## Experience Notes Path: `{working-directory}/browser-act-skill-forge-memories/xianyu-scraper-goofish-search-list.memory.md` **Before execution**: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly. **After execution**: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: `{YYYY-MM-DD}: {what happened} → {conclusion}` Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.