Scrape a web page into JSON

Scraping a page by hand is fragile: you write selectors, the page changes, it breaks. Firecrawl flips it around: you tell it what data you want in plain language and an agent reads the page and hands it back as JSON. We'll pull a product list off a shop, but it works for any data: prices, listings, directories, profiles.

Why this holds up where a normal scraper doesn't: a hand-written scraper depends on the page's exact structure, so the day they rename a class, it breaks. The agent reads the page the way you would, by meaning. Change the layout and it still finds the price, because you asked for "the price," not for div.col-3 > span.amount.

1. Install the CLI and sign in

Firecrawl has a command-line tool. Install it with Node and authenticate:

npm install -g firecrawl-cli

Check it's ready. If you're missing the key, the command walks you through creating it at firecrawl.dev (the free plan gives you plenty of credits to start). Store it in a .env like any other key, never in the code: it's covered in your first API key, without leaking it.

firecrawl --status
● Authenticated
  Credits: 500,000 remaining

2. First, see what's on the page

Before extracting anything, scrape the page to see its clean content. This hands back the page in markdown, no menus, no noise:

firecrawl scrape https://example-shop.com/products -o page.md

Open page.md and check the data you want is in there. If it shows up, the agent will be able to extract it.

3. Ask for the data as JSON

Now the good part. The agent command takes a sentence with what you want and the URL, and hands back JSON. --wait waits for it to finish and --pretty makes it readable:

firecrawl agent "extract each product with its name and price" \
  --urls https://example-shop.com/products \
  --wait --pretty -o products.json

And products.json comes out like this, ready to use anywhere:

[
  { "name": "Nordic Chair",  "price": 89.00 },
  { "name": "Oak Table",     "price": 240.00 },
  { "name": "Floor Lamp",    "price": 65.50 }
]

4. Want more precision? Give it a schema

The sentence is enough for most cases. If you need the fields to come out exact every time (same names, same types), pass a schema with --schema:

firecrawl agent "extract the products" \
  --urls https://example-shop.com/products \
  --schema '{"type":"array","items":{"type":"object","properties":{"name":{"type":"string"},"price":{"type":"number"}}}}' \
  --wait --pretty -o products.json

When something breaks

The usual snags, and the fix for each:

• "command not found: firecrawl". Node isn't installed, or the global install didn't land. Check install Node, then run the install line again.
• 401 or "out of credits". The key isn't set, or you've spent the free credits. Store it the right way (your first API key, without leaking it) and check your dashboard.
• The JSON is missing fields. The sentence was too vague. Name the fields exactly, or pass a --schema like in step 4.
• It returned nothing. The data loads after a click or a login, so it wasn't on the page the agent read. Open the page.md from step 2: if it's not in there, the agent can't see it either.

Now try this

Same command, more reach:

• Several pages at once. Pass more than one URL to --urls and it merges them into one file.
• Watch a competitor's prices. Point it at their shop, keep the JSON, and diff it next week.
• Make it run on its own. Drop this into a scheduled task and get fresh data every morning without lifting a finger.
• Feed it somewhere. JSON drops straight into a sheet, a database, or the next step of an automation.

What you just did

You turned a web page into data you can use, without writing or maintaining a scraper. What you asked for was products, but the same command pulls a competitor's prices, a directory's listings, or any data that shows up on a page. The work stopped being coding the scraper; now it's just saying what you want.