Shopify Store Scraper — Products, Prices, Variants and Store Data

What it does

Most Shopify stores expose their published catalog as JSON at /products.json. This actor reads that feed for one or more store URLs and returns one row per product: title, handle, vendor, product type, tags, created, published and updated dates, price range, compare-at price and discount percentage, stock availability, every variant (SKU, price, compare-at, options, weight) and every image URL.

For competitor price monitoring, pass updatedSince on a daily schedule and only products updated since that date become rows, so unchanged products cost nothing. collections narrows a run to specific collection handles. In store mode you get one row per shop instead: name, country, currency, ships-to countries, product and collection counts, price statistics, top vendors, product types and tags, the biggest collections, the theme and the apps detected on the homepage.

It is polite by design: the store's robots.txt and Crawl-delay are obeyed, requests go one at a time per store at about 1.3 per second, and any block stops that store with a free row. Products cost $0.30 per 1,000 and each analyzed store $0.005; blocked, headless and non-Shopify sites are free.

How it works

1. Each URL in storeUrls (or storeUrl) is reduced to its host, so any page of a store works, including myshopify.com domains.
2. The actor reads the store's robots.txt first, then pages through products.json at 250 products per page, up to maxProductsPerStore.
3. Filters are applied as it goes: collections switches to collection feeds and updatedSince keeps only recently updated products, which are the only ones charged.
4. In store or both mode it also reads meta.json and the homepage HTML to compute price stats and detect the theme and apps from open fingerprints.
5. A 401, 403, 429 or bot challenge stops that store at once with no retries, proxies or bypass, and the resulting row is free.

Input example

A typical run input (JSON). Every field is also available as a form in the Apify console.

{
  "storeUrls": [
    "allbirds.com",
    "https://www.tentree.com"
  ],
  "mode": "products",
  "collections": [],
  "maxProductsPerStore": 1000,
  "updatedSince": "",
  "includeVariants": true,
  "includeBodyHtml": false
}

Output example

Product rows carry the store, pricing, discount and stock fields up front, with variants and images attached to each row:

Pricing

Pay per event: you are charged only for the results you receive, with no subscription. Set a maximum total charge on any run and it stops cleanly at the cap.

Prices are read from the live Apify Store listing when this page is built. The listing is always authoritative.

Use cases

How it compares

If you own the store, Shopify's admin export includes inventory quantities and data this public feed never shows. For one small store, opening /products.json in a browser may be all you need; the actor helps once you have many stores, schedules or large catalogs.

Limits and honest notes

• Only stores that serve Shopify's public /products.json work. Headless storefronts (Hydrogen/Oxygen or a custom frontend, e.g. fashionnova.com, hydrogen.shop) do not serve it and come back as free not-reachable rows.
• Each store's own robots.txt is read first and obeyed, including any Crawl-delay. The actor never requests /search, /cart, /checkout, /policies/, /recommendations/products or sort_by collection URLs. A store that disallows products.json gives a free robots-disallowed row.
• Polite by design: one request at a time per store at about 1.3 requests/second, at most 3 stores in parallel, honest user agent (ActorForge-ShopifyScraper/1.0). A 401/403/429 or bot challenge stops that store at once: no retries, proxies or bypass. Some big brands (e.g. gymshark.com with a 403, bombas.com with a Vercel challenge) block this, so their rows are free.
• products.json returns 250 products per page and lists published products only. maxProductsPerStore caps each store (default 1,000, up to 100,000). In both mode the store stats cover the products scanned, and catalogComplete says whether that was the whole catalogue.
• Prices are the store's base-currency prices from products.json (meta.json gives the currency). Market-specific or B2B prices, inventory quantities and sales figures are not public there.
• Apps and theme come from the homepage HTML (Shopify.theme and the open webappanalyzer fingerprints). Apps that load only after JavaScript runs or only on product pages may be missed.
• Catalogue data only: no customer, reviewer or contact-person data is read or returned.
• The run's spend limit is respected: products are charged one page at a time before delivery, and the run stops cleanly when the limit is reached.

FAQ

Use it from code

Run the actor through the Apify API with the input above.

cURL

curl -X POST "https://api.apify.com/v2/acts/actor-forge~shopify-store-scraper/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"storeUrls": ["allbirds.com", "https://www.tentree.com"], "mode": "products", "collections": [], "maxProductsPerStore": 1000, "updatedSince": "", "includeVariants": true, "includeBodyHtml": false}'

JavaScript (Node.js)

// Node.js 18+ (built-in fetch). Set APIFY_TOKEN in your environment.
const input = {
  "storeUrls": [
    "allbirds.com",
    "https://www.tentree.com"
  ],
  "mode": "products",
  "collections": [],
  "maxProductsPerStore": 1000,
  "updatedSince": "",
  "includeVariants": true,
  "includeBodyHtml": false
};

const res = await fetch(
  "https://api.apify.com/v2/acts/actor-forge~shopify-store-scraper/run-sync-get-dataset-items",
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.APIFY_TOKEN}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify(input),
  }
);
const items = await res.json();
console.log(items.length, items[0]);

Python

# pip install apify-client   (set APIFY_TOKEN in your environment)
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run_input = {
    "storeUrls": [
        "allbirds.com",
        "https://www.tentree.com"
    ],
    "mode": "products",
    "collections": [],
    "maxProductsPerStore": 1000,
    "updatedSince": "",
    "includeVariants": True,
    "includeBodyHtml": False
}

run = client.actor("actor-forge/shopify-store-scraper").call(run_input=run_input)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

The run-sync-get-dataset-items endpoint waits for the run (up to 5 minutes) and returns the dataset rows directly. For longer runs, start the run asynchronously and read the dataset when it finishes, as the Python example does. Replace APIFY_TOKEN with your own token from the Apify console.

Step-by-step guides

Related actors

More E-commerce & App Stores tools · All actors