How to scrape Shopify products (prices, variants and stock) to CSV with Python

By Ryu Moon

If you sell online, sooner or later you want to see a competitor's whole catalogue in one table: what they sell, what it costs, what is on sale and what has sold out. When that competitor runs on Shopify, you do not need a headless browser or a login to get it. This guide shows how to export every product from any Shopify store, or from a single collection, into CSV with a few lines of Python. It uses Shopify Products Scraper – Prices, Variants, Stock, an Apify Actor I publish.

How Shopify product scraping works

Most Shopify storefronts serve a public product feed at /products.json. It is the same data the store's own pages use, and it contains each product's title, vendor, product type, tags, images, timestamps and every variant with its SKU, price, compare-at price and an "available" flag.

You can check whether a site is on Shopify in about five seconds. Add /products.json to the store's address in your browser:

https://colourpop.com/products.json?limit=5

If you see JSON with a products array, the store works. When I tried this on 10 November 2026, colourpop.com/products.json?limit=250 returned a full page of 250 products. The same request to another well-known Shopify brand returned HTTP 429 (Too Many Requests) from my test machine. That is the real-world catch: the feed is public, but stores rate-limit, paginate and sometimes block, and doing it yourself means handling pagination, retries, collections and messy price fields across every store.

The Actor handles that part. You give it store or collection URLs, and it returns one clean row per product.

What you get per product

Each dataset row includes:

• storeName, currency, title, vendor, productType, tags, url
• priceMin, priceMax, compareAtPriceMax, onSale and maxDiscountPercent
• available, variantsCount, variantsAvailable
• variants: every size or colour with sku, price, compareAtPrice and available
• imageUrl plus all images, descriptionText, and createdAt, publishedAt, updatedAt

A run summary (products scanned and saved, plus any stores that failed or are not on Shopify) is saved as a SUMMARY record in the run's key-value store.

What it does not give you, because Shopify's public feed does not include it: exact stock quantities (only in stock or not), sales numbers, or prices in other countries' local currencies. It also only works on Shopify. WooCommerce, Magento and BigCommerce sites are reported in SUMMARY and are not charged.

Option 1: run it in the Apify Console (no code)

1. Open apify.com/ryumoon-data/shopify-store-products-scraper and open it in Apify Console (a free Apify account is enough to try it).
2. In Shopify store or collection URLs, paste store home pages (https://colourpop.com) or collection pages (https://www.allbirds.com/collections/mens). Plain domains are fine too.
3. Set Max products per store to something small, such as 20, for your first run. 0 means the whole catalogue.
4. Optionally turn on filters: In-stock products only, On-sale products only, a price range, title keywords, vendors or product types.
5. Click Start, then open the Output tab and export as CSV, JSON or Excel.

Option 2: Python with apify-client

Install the official client:

pip install "apify-client>=3"

Get your API token from Apify Console → Settings → API & Integrations and put it in an environment variable rather than in your code:

export APIFY_TOKEN="your-own-token-here"

This script exports a store's products, with one row per product, to products.csv:

import csv
import os
from decimal import Decimal

from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run_input = {
    "startUrls": [
        {"url": "https://colourpop.com"},
        {"url": "https://www.allbirds.com/collections/mens"},
    ],
    "maxProductsPerStore": 200,   # 0 = whole catalogue
    "onlyAvailable": False,
    "onlyOnSale": False,
    "includeVariants": True,
    "includeImages": False,       # smaller output
    "includeDescription": False,
}

run = client.actor("ryumoon-data/shopify-store-products-scraper").call(
    run_input=run_input,
    max_total_charge_usd=Decimal("1.00"),  # hard spending cap for this run
)
print("Run status:", run.status)

columns = [
    "storeName", "title", "vendor", "productType", "priceMin", "priceMax",
    "compareAtPriceMax", "currency", "onSale", "maxDiscountPercent",
    "available", "variantsCount", "url", "updatedAt",
]

with open("products.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(f, fieldnames=columns, extrasaction="ignore")
    writer.writeheader()
    for item in client.dataset(run.default_dataset_id).iterate_items():
        writer.writerow(item)

summary = client.key_value_store(run.default_key_value_store_id).get_record("SUMMARY")
if summary:
    print(summary["value"])

A few notes:

• startUrls is the only required field. Each entry is an object with a url.
• A store URL gets the whole catalogue. A /collections/... URL gets just that collection.
• max_total_charge_usd tells Apify to stop charging once the run reaches that amount. Together with maxProductsPerStore, it keeps a test run cheap.
• The script uses apify-client 3.x, where call() returns a Run object (run.default_dataset_id). On the older 1.x/2.x clients, use run["defaultDatasetId"] instead.

Variant-level CSV (one row per SKU)

If you need sizes and colours as separate rows, for example to track which SKUs sell out, flatten the variants array:

with open("variants.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.writer(f)
    writer.writerow(["store", "product", "sku", "variant", "price",
                     "compareAtPrice", "available", "url"])
    for item in client.dataset(run.default_dataset_id).iterate_items():
        for v in item.get("variants") or []:
            writer.writerow([
                item.get("storeName"), item.get("title"), v.get("sku"),
                v.get("title"), v.get("price"), v.get("compareAtPrice"),
                v.get("available"), item.get("url"),
            ])

Monitor competitor prices and sales daily

For ongoing monitoring, you rarely need the full catalogue every day. Two inputs keep daily runs small:

• updatedWithinDays: 1 returns only products the store changed in the last day.
• onlyOnSale: true returns only products where a variant's price is below its compare-at price.

monitor_input = {
    "startUrls": [{"url": "https://colourpop.com"}],
    "updatedWithinDays": 1,
    "onlyOnSale": True,
    "minPrice": 10,                 # in the store's own currency; 0 = no minimum
    "titleKeywords": ["lip", "palette"],
    "includeVariants": True,
}

run = client.actor("ryumoon-data/shopify-store-products-scraper").call(
    run_input=monitor_input,
)
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(f'{item["title"]}: {item["priceMin"]} {item["currency"]} '
          f'(up to {item["maxDiscountPercent"]}% off)')

To run it on a timer, save the input as a Task in the Apify Console and attach a Schedule (for example daily at 07:00). Each run writes a new dataset that you can pull with the same code, or send to Google Sheets, Make, Zapier or n8n.

Other filters in the input schema: maxPrice, vendors (exact vendor names, useful on multi-brand stores), productTypes (for example "Hoodie"), and maxConcurrency (how many stores are scraped in parallel, 1–10, default 3).

When a store returns 403 or 429

Some stores rate-limit or block plain requests, like the 429 I hit above. If a store fails, the SUMMARY record will say so. Then:

1. Retry with fewer stores in parallel ("maxConcurrency": 1).
2. Enable Apify Proxy, which you can do from the input:

run_input["proxyConfiguration"] = {
    "useApifyProxy": True,
    "apifyProxyGroups": ["RESIDENTIAL"],
}

Proxy traffic is billed to your Apify account separately from the Actor's own price. Password-protected stores cannot be read at all.

Very large catalogues are a known limit of the public feed. The Actor's README says stores above about 25,000 products may be incomplete. For those, scrape collection by collection.

What it costs

The Actor uses pay-per-event pricing. On its Store page today (10 November 2026), that is $0.0008 per product saved ($0.80 per 1,000 products), plus a $0.0001 Actor start event per GB of memory. Each product includes all its variants and images, so a 50-variant product still counts once. Filters run before you are charged, so products removed by onlyOnSale, a price range or updatedWithinDays cost nothing. Check the pricing tab before a big run, since prices can change. Proxy usage is extra, as above.

Is it OK to scrape Shopify stores?

The Actor reads the product feed the storefront already serves publicly to every visitor. It collects product data only: no customer data, no personal data and no logins. You are still responsible for how you use the data. Comparing prices is one thing. Republishing a store's photos and descriptions as your own is another, and respecting the store's terms and intellectual property is on you.

Summary

• Add /products.json to a store's URL to check it is on Shopify.
• Run Shopify Products Scraper with startUrls and a small maxProductsPerStore first.
• Pull the dataset into CSV with apify-client, using your own APIFY_TOKEN from an environment variable.
• For monitoring, combine updatedWithinDays, onlyOnSale and an Apify Schedule.

If something breaks on a particular store, open an issue on the Actor's Issues tab with the store URL.