# How to scrape Shopify products (prices, variants and stock) to CSV with Python

*By Ryu Moon*

If you sell online, sooner or later you want to see a competitor's whole catalogue in one table: what they sell, what it costs, what is on sale and what has sold out. When that competitor runs on Shopify, you do not need a headless browser or a login to get it. This guide shows how to export every product from any Shopify store, or from a single collection, into CSV with a few lines of Python. It uses [Shopify Products Scraper – Prices, Variants, Stock](https://apify.com/ryumoon-data/shopify-store-products-scraper), an Apify Actor I publish.

## How Shopify product scraping works

Most Shopify storefronts serve a public product feed at `/products.json`. It is the same data the store's own pages use, and it contains each product's title, vendor, product type, tags, images, timestamps and every variant with its SKU, price, compare-at price and an "available" flag.

You can check whether a site is on Shopify in about five seconds. Add `/products.json` to the store's address in your browser:

```
https://colourpop.com/products.json?limit=5
```

If you see JSON with a `products` array, the store works. When I tried this on 10 November 2026, `colourpop.com/products.json?limit=250` returned a full page of 250 products. The same request to another well-known Shopify brand returned **HTTP 429 (Too Many Requests)** from my test machine. That is the real-world catch: the feed is public, but stores rate-limit, paginate and sometimes block, and doing it yourself means handling pagination, retries, collections and messy price fields across every store.

The Actor handles that part. You give it store or collection URLs, and it returns one clean row per product.

## What you get per product

Each dataset row includes:

- `storeName`, `currency`, `title`, `vendor`, `productType`, `tags`, `url`
- `priceMin`, `priceMax`, `compareAtPriceMax`, `onSale` and `maxDiscountPercent`
- `available`, `variantsCount`, `variantsAvailable`
- `variants`: every size or colour with `sku`, `price`, `compareAtPrice` and `available`
- `imageUrl` plus all `images`, `descriptionText`, and `createdAt`, `publishedAt`, `updatedAt`

A run summary (products scanned and saved, plus any stores that failed or are not on Shopify) is saved as a `SUMMARY` record in the run's key-value store.

What it does **not** give you, because Shopify's public feed does not include it: exact stock quantities (only in stock or not), sales numbers, or prices in other countries' local currencies. It also only works on Shopify. WooCommerce, Magento and BigCommerce sites are reported in `SUMMARY` and are not charged.

## Option 1: run it in the Apify Console (no code)

1. Open [apify.com/ryumoon-data/shopify-store-products-scraper](https://apify.com/ryumoon-data/shopify-store-products-scraper) and open it in Apify Console (a free Apify account is enough to try it).
2. In **Shopify store or collection URLs**, paste store home pages (`https://colourpop.com`) or collection pages (`https://www.allbirds.com/collections/mens`). Plain domains are fine too.
3. Set **Max products per store** to something small, such as 20, for your first run. `0` means the whole catalogue.
4. Optionally turn on filters: **In-stock products only**, **On-sale products only**, a price range, title keywords, vendors or product types.
5. Click **Start**, then open the **Output** tab and export as CSV, JSON or Excel.

## Option 2: Python with apify-client

Install the official client:

```bash
pip install "apify-client>=3"
```

Get your API token from **Apify Console → Settings → API & Integrations** and put it in an environment variable rather than in your code:

```bash
export APIFY_TOKEN="your-own-token-here"
```

This script exports a store's products, with one row per product, to `products.csv`:

```python
import csv
import os
from decimal import Decimal

from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run_input = {
    "startUrls": [
        {"url": "https://colourpop.com"},
        {"url": "https://www.allbirds.com/collections/mens"},
    ],
    "maxProductsPerStore": 200,   # 0 = whole catalogue
    "onlyAvailable": False,
    "onlyOnSale": False,
    "includeVariants": True,
    "includeImages": False,       # smaller output
    "includeDescription": False,
}

run = client.actor("ryumoon-data/shopify-store-products-scraper").call(
    run_input=run_input,
    max_total_charge_usd=Decimal("1.00"),  # hard spending cap for this run
)
print("Run status:", run.status)

columns = [
    "storeName", "title", "vendor", "productType", "priceMin", "priceMax",
    "compareAtPriceMax", "currency", "onSale", "maxDiscountPercent",
    "available", "variantsCount", "url", "updatedAt",
]

with open("products.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(f, fieldnames=columns, extrasaction="ignore")
    writer.writeheader()
    for item in client.dataset(run.default_dataset_id).iterate_items():
        writer.writerow(item)

summary = client.key_value_store(run.default_key_value_store_id).get_record("SUMMARY")
if summary:
    print(summary["value"])
```

A few notes:

- `startUrls` is the only required field. Each entry is an object with a `url`.
- A store URL gets the whole catalogue. A `/collections/...` URL gets just that collection.
- `max_total_charge_usd` tells Apify to stop charging once the run reaches that amount. Together with `maxProductsPerStore`, it keeps a test run cheap.
- The script uses apify-client 3.x, where `call()` returns a `Run` object (`run.default_dataset_id`). On the older 1.x/2.x clients, use `run["defaultDatasetId"]` instead.

## Variant-level CSV (one row per SKU)

If you need sizes and colours as separate rows, for example to track which SKUs sell out, flatten the `variants` array:

```python
with open("variants.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.writer(f)
    writer.writerow(["store", "product", "sku", "variant", "price",
                     "compareAtPrice", "available", "url"])
    for item in client.dataset(run.default_dataset_id).iterate_items():
        for v in item.get("variants") or []:
            writer.writerow([
                item.get("storeName"), item.get("title"), v.get("sku"),
                v.get("title"), v.get("price"), v.get("compareAtPrice"),
                v.get("available"), item.get("url"),
            ])
```

## Monitor competitor prices and sales daily

For ongoing monitoring, you rarely need the full catalogue every day. Two inputs keep daily runs small:

- `updatedWithinDays: 1` returns only products the store changed in the last day.
- `onlyOnSale: true` returns only products where a variant's price is below its compare-at price.

```python
monitor_input = {
    "startUrls": [{"url": "https://colourpop.com"}],
    "updatedWithinDays": 1,
    "onlyOnSale": True,
    "minPrice": 10,                 # in the store's own currency; 0 = no minimum
    "titleKeywords": ["lip", "palette"],
    "includeVariants": True,
}

run = client.actor("ryumoon-data/shopify-store-products-scraper").call(
    run_input=monitor_input,
)
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(f'{item["title"]}: {item["priceMin"]} {item["currency"]} '
          f'(up to {item["maxDiscountPercent"]}% off)')
```

To run it on a timer, save the input as a **Task** in the Apify Console and attach a **Schedule** (for example daily at 07:00). Each run writes a new dataset that you can pull with the same code, or send to Google Sheets, Make, Zapier or n8n.

Other filters in the input schema: `maxPrice`, `vendors` (exact vendor names, useful on multi-brand stores), `productTypes` (for example `"Hoodie"`), and `maxConcurrency` (how many stores are scraped in parallel, 1–10, default 3).

## When a store returns 403 or 429

Some stores rate-limit or block plain requests, like the 429 I hit above. If a store fails, the `SUMMARY` record will say so. Then:

1. Retry with fewer stores in parallel (`"maxConcurrency": 1`).
2. Enable Apify Proxy, which you can do from the input:

```python
run_input["proxyConfiguration"] = {
    "useApifyProxy": True,
    "apifyProxyGroups": ["RESIDENTIAL"],
}
```

Proxy traffic is billed to your Apify account separately from the Actor's own price. Password-protected stores cannot be read at all.

Very large catalogues are a known limit of the public feed. The Actor's README says stores above about 25,000 products may be incomplete. For those, scrape collection by collection.

## What it costs

The Actor uses pay-per-event pricing. On its Store page today (10 November 2026), that is **$0.0008 per product saved ($0.80 per 1,000 products)**, plus a $0.0001 Actor start event per GB of memory. Each product includes all its variants and images, so a 50-variant product still counts once. Filters run before you are charged, so products removed by `onlyOnSale`, a price range or `updatedWithinDays` cost nothing. Check the pricing tab before a big run, since prices can change. Proxy usage is extra, as above.

## Is it OK to scrape Shopify stores?

The Actor reads the product feed the storefront already serves publicly to every visitor. It collects product data only: no customer data, no personal data and no logins. You are still responsible for how you use the data. Comparing prices is one thing. Republishing a store's photos and descriptions as your own is another, and respecting the store's terms and intellectual property is on you.

## Summary

- Add `/products.json` to a store's URL to check it is on Shopify.
- Run [Shopify Products Scraper](https://apify.com/ryumoon-data/shopify-store-products-scraper) with `startUrls` and a small `maxProductsPerStore` first.
- Pull the dataset into CSV with `apify-client`, using your own `APIFY_TOKEN` from an environment variable.
- For monitoring, combine `updatedWithinDays`, `onlyOnSale` and an Apify Schedule.

If something breaks on a particular store, open an issue on the Actor's Issues tab with the store URL.