# Build And Deploy A Multi-Source App Demand Validator

## Overview

This workflow builds a Blocks Network provider agent that turns an app idea into an evidence-backed Markdown demand report. It combines search and app-store evidence from DataForSEO, public social conversation from Apify, and website-market evidence from Firecrawl, then uses OpenAI to plan queries and synthesize the findings.

The operator configures these services once on the hosted agent. A team can then use one shared web page or Blocks agent without every student or teammate opening and maintaining separate DataForSEO, Apify, Firecrawl, and OpenAI accounts. The operator still owns provider billing, access policy, quotas, and compliance.

## When To Use This

- Help a class, innovation team, product group, or accelerator compare early app ideas using consistent evidence.
- Check whether a problem shows search demand, existing mobile adoption, active public discussion, or competing web services before building.
- Give non-technical users one controlled research interface while centralizing external-tool credentials and spending.
- Produce a repeatable Markdown artifact that separates measured facts, interpretations, recommendations, and limitations.

## Prerequisites

- Node.js 20 or newer and npm.
- A Blocks account, a `BLOCKS_API_KEY`, provider registration permission, and the Blocks CLI/runtime available through `@blocks-network/sdk`.
- An OpenAI account with API billing and an `OPENAI_API_KEY` for query planning, social classification, and report synthesis.
- A DataForSEO account with funded API access plus `DATAFORSEO_USERNAME` and `DATAFORSEO_PASSWORD` when SEO or app-store research is enabled.
- An Apify account and `APIFY_TOKEN` when social research is enabled. Confirm that the configured Reddit, YouTube, and TikTok Actors remain available and that their input schemas still match the adapters.
- A Firecrawl account and `FIRECRAWL_API_KEY` when website research is enabled. Set the request rate to the allowance shown by the account plan.
- A Railway account, project access, and authenticated Railway CLI for the long-running provider.
- Optional: a Netlify account and authenticated CLI to publish the included classroom-friendly frontend.
- GitHub access only if the source or documentation will be published.

End users do not need accounts with the individual research providers. They only need permission to use the shared frontend or Blocks agent. Never send provider secrets to the browser or include them in reports.

## Required Inputs

The agent accepts one JSON request part. `idea` is required; all other fields have defaults.

```json
{
  "idea": "ShadeRoute maps shade-optimized walking and biking routes using canopy data and crowdsourced reports.",
  "targetCustomer": "Urban pedestrians, cyclists, heat-vulnerable residents, and city planners",
  "primaryMarket": "United States",
  "language": "English",
  "platforms": ["ios", "android"],
  "businessModel": "",
  "knownCompetitors": [],
  "dataForSeoEnabled": true,
  "apifyEnabled": false,
  "firecrawlEnabled": false,
  "socialSources": ["reddit", "youtube", "tiktok"],
  "analysisDepth": "light",
  "maxSocialCostUsd": 2
}
```

Rules:

- Enable at least one of `dataForSeoEnabled`, `apifyEnabled`, or `firecrawlEnabled`.
- Use at least one platform: `ios` or `android`.
- When Apify is enabled, choose at least one social source and use a social budget from USD 1.50 to USD 20.
- Default to `light` depth for classroom use. `standard` and `deep` make more provider calls and cost more.

## Tools And Services

- **Blocks Network:** exposes the workflow as reusable agent `app_demand_validator`, handles task progress, concurrency, and the Markdown artifact.
- **DataForSEO:** measures Google Ads keyword volume, CPC, competition, organic SERPs, Apple App Store results, Google Play results, and Google Play lifetime-install evidence when available. Lifetime installs are not monthly downloads.
- **Apify:** runs public-data Actors for Reddit, YouTube, and TikTok. It helps estimate how often the problem appears in observed conversations, not the number of potential customers.
- **Firecrawl:** searches for and reads websites, organizations, tools, and services already addressing the problem.
- **OpenAI:** converts an idea into search, commercial, app-store, social, and web queries; classifies social relevance; and writes evidence-grounded report sections.
- **Railway:** hosts the persistent Blocks provider process and its server-side secrets.
- **Netlify:** optionally hosts the web form and serverless bridge to Blocks. The browser never receives provider credentials.

## Workflow

### 1. Scaffold The Provider

Create a Node/TypeScript project with this core layout:

```text
agent-card.json
handler.ts
trigger.ts
src/
  input.ts
  openai.ts
  dataforseo.ts
  apify.ts
  firecrawl.ts
  report-renderer.ts
web/
netlify/functions/
railway.json
netlify.toml
```

Install the runtime packages:

```bash
npm install @blocks-network/sdk apify-client dotenv openai
npm install --save-dev @types/node tsx typescript
```

Configure package scripts for `blocks-run`, TypeScript validation, Node tests, Blocks schema checks, and a hosted trigger.

### 2. Define The Blocks Contract

In `agent-card.json`:

- Use agent name `app_demand_validator`.
- Accept an `application/json` request using the fields shown above.
- Guarantee one `text/markdown` output named `demand_report` with filename `app-demand-report.md`.
- Use a runtime concurrency of 10, expected instance count of 1, pending backlog of 30, and maximum running time of 600 seconds.

Keep the source toggles independent. DataForSEO should default on; Apify and Firecrawl should default off; analysis depth should default to light.

### 3. Keep Credentials On The Server

Create `.env` from `.env.example` and keep `.env` ignored by Git:

```dotenv
BLOCKS_API_KEY=
OPENAI_API_KEY=
OPENAI_MODEL=gpt-4.1-mini
DATAFORSEO_USERNAME=
DATAFORSEO_PASSWORD=
APIFY_TOKEN=
FIRECRAWL_API_KEY=
FIRECRAWL_REQUESTS_PER_MINUTE=5
APIFY_REDDIT_ACTOR_ID=prodiger/reddit-scraper
APIFY_YOUTUBE_ACTOR_ID=streamers/youtube-scraper
APIFY_TIKTOK_ACTOR_ID=clockworks/tiktok-scraper
```

Only the operator should set these values locally and in Railway. The web frontend should call a server-side bridge that uses `BLOCKS_API_KEY`; it must not call research providers directly.

### 4. Plan Demand Queries

Use OpenAI with structured output to generate five query groups:

- Search keywords for topic demand.
- Commercial-intent queries for solution-seeking demand.
- App-store keywords for mobile competitors.
- Social queries describing the problem in users' language.
- Web queries for existing products, organizations, and services.

Include the exact generated queries in the final report. Query visibility lets reviewers challenge the methodology and distinguish a weak market from a weak search phrase.

### 5. Collect Independent Evidence Streams

Run enabled providers concurrently with `Promise.all`, but isolate their failures.

For DataForSEO:

- Batch keyword-volume requests.
- Collect organic Google SERPs.
- submit Apple and Google app searches as priority tasks, poll until ready, and enrich a limited number of Google Play apps with app-info results.
- Record missing metrics as `Not reported`, never as zero.

For Apify:

- Split `maxSocialCostUsd` across enabled sources and pass the per-source value as `maxTotalChargeUsd`.
- Limit results by research depth.
- Normalize URLs, text, dates, engagement, and hashed author identities.
- Use OpenAI to classify relevance, pain intensity, and solution-seeking intent before calculating summary counts.
- Report observed public items and observed authors, not market size.

For Firecrawl:

- Call `POST https://api.firecrawl.dev/v2/search` with web search enabled and Markdown main-content extraction.
- Use one query and six results at light depth, two and eight at standard, and three and ten at deep.
- Serialize request starts using `FIRECRAWL_REQUESTS_PER_MINUTE`. Clamp the local setting to a safe range, but never set it higher than the account's vendor allowance because that will produce HTTP 429 responses.
- Deduplicate returned URLs and track Firecrawl credits when reported.

Each adapter should return evidence plus source status. A failed or disabled source must be visible in the report while successful sources continue.

### 6. Render A Stable Markdown Report

Have OpenAI return content for a strict section schema, then use deterministic code to control headings, tables, and order. Produce these sections:

1. Executive verdict
2. Idea and audience
3. Demand scorecard
4. DataForSEO research - SEO and mobile apps
5. Apify research - social conversation
6. Firecrawl research - website landscape
7. Actual demand queries used
8. Overall insights and recommendations
9. Methodology and limitations

Always show each provider section. If its toggle is off, mark it as skipped. Under DataForSEO, render a deterministic keyword table with monthly volume, CPC, competition, and recent monthly history. Under Firecrawl, link the websites found and show the query that discovered each one.

### 7. Add The Shared Frontend

Create a form for the idea, audience, market, platforms, research depth, and three provider toggles. Use these defaults:

- DataForSEO on.
- Apify off.
- Firecrawl off.
- Light depth selected.

Explain that up to 10 classroom requests can run simultaneously and that additional tasks wait in the provider backlog. The serverless start function validates the request and sends it to Blocks; a status function lets the browser poll for progress and retrieve the Markdown artifact.

This shared layer is what removes individual provider onboarding for the team. Users operate one approved interface while service credentials, budgets, and throttling remain centralized.

### 8. Validate Locally

Run:

```bash
npm run typecheck
npm test
npm run check
```

Test input defaults and validation, Firecrawl normalization and deduplication, social summarization, query rendering, source-skipped behavior, keyword-table output, website-table output, and that `Methodology and limitations` is the final section.

Start the provider locally with `npm start`. In another terminal, set `TRIGGER_SOURCES` to `dataforseo`, `apify`, `firecrawl`, or `all`, then run `npm run trigger`. Begin with light, single-source smoke tests to control cost.

### 9. Deploy And Register

Authenticate the Railway and Blocks CLIs. Create or link the Railway project, set all required variables on the Railway service, and deploy:

```bash
railway up
railway deployment list
blocks register --no-input
```

Confirm Railway logs show the provider starting and registering. Then trigger the hosted agent and save its artifact. The deployed Blocks app is:

https://app.blocks.ai/agents/app_demand_validator

If publishing the frontend, set its `BLOCKS_API_KEY` in Netlify's server environment and deploy the `web` directory plus functions. The finished web interface for this implementation is:

https://demand-signal-validator.netlify.app

## Outputs

- `app-demand-report.md`, an evidence-backed Markdown report with source-specific sections, actual demand queries, overall recommendations, and limitations.
- Source-status records showing success, failure, skipped coverage, item counts, costs, and credits when available.
- A shareable Blocks agent and optional web interface for team use.

## Validation

- `npm run typecheck`, `npm test`, and `npm run check` all pass.
- A local light-mode single-source trigger produces one Markdown artifact.
- A hosted Blocks trigger completes and downloads `app-demand-report.md`.
- The live frontend shows the three independent toggles and defaults.
- A request with every source disabled is rejected.
- The report lists the exact queries, labels missing values correctly, and ends with `Methodology and limitations`.
- Railway reports a successful deployment and the agent is reachable at its Blocks URL.

## Safety And Quality Notes

- Central access simplifies team use but does not remove the operator's responsibility for provider terms, privacy, billing, and user authorization.
- Never expose API keys in frontend code, logs, screenshots, reports, examples, Git history, or Blocks artifacts.
- Apply strict budgets, timeouts, depth limits, and rate limits. Ten concurrent agent tasks do not imply that every upstream provider can accept ten simultaneous calls.
- Public social posts are directional evidence. Hash author identities, avoid unnecessary personal data, and do not equate posts, views, or engagement with customers.
- App-store install counts are lifetime ranges or provider-derived counts when available, not current download velocity.
- Web pages and search results show existing supply and language, not automatically validated demand.
- Keep measured evidence separate from modeled interpretations. Missing evidence should lower confidence, not be filled with invented metrics.
- Third-party APIs and Apify Actor schemas can change. Fail sources independently, record the limitation, and retest adapters before group sessions.
- Start classroom sessions in light mode with Apify and Firecrawl off, then enable additional evidence only when needed and budgeted.