By Ryu Moon
RSS is still everywhere. Most blogs, news sites, podcasts, GitHub release pages and every YouTube channel publish a feed that software can read. The problem is that feeds come in several formats (RSS 2.0, RSS 1.0/RDF, Atom, JSON Feed, podcast feeds), dates are written in a dozen styles, and content arrives as HTML. If you want the data in a spreadsheet, a database or an AI pipeline, you usually want plain JSON.
People search for exactly this: "rss to json python", "rss to json api", "convert rss feed to json", "how to parse rss feed in python" and "rss feed to csv" all appear in Google's autocomplete. This guide shows the quick do-it-yourself way in Python, where it starts to hurt, and how to convert many feeds (or just website addresses) to clean JSON with one API call.
Step 1: The quick way with feedparser
For one feed, the feedparser library is the classic answer:
pip install feedparser
import json
import feedparser
feed = feedparser.parse("https://hnrss.org/frontpage")
items = []
for entry in feed.entries[:5]:
items.append({
"title": entry.get("title"),
"link": entry.get("link"),
"published": entry.get("published"),
"summary": entry.get("summary", "")[:200],
})
print(json.dumps(items, indent=2))
I ran this on the Hacker News front page feed on 17 Nov 2026 and it worked first time. Look at the output, though:
{
"title": "Mem Is Shutting Down",
"link": "https://get.mem.ai/blog/mem-is-shutting-down",
"published": "Tue, 17 Nov 2026 02:13:41 +0000",
"summary": "<p>Article URL: <a href=\"https://get.mem.ai/blog/mem-is-shutting-down\">..."
}
Two things you will have to fix yourself straight away:
- Dates are strings in the feed's own format. Here it's RFC 822 (
Tue, 17 Nov 2026 02:13:41 +0000). Other feeds use ISO 8601, time-zone names likeESTorBST, or no date at all. feedparser gives you apublished_parsedtuple when it can, but you still need to turn it into something you can sort and store. - Content is HTML. Summaries and content are full of tags, tracking pixels and relative links. For a spreadsheet or an LLM prompt you want plain text or Markdown.
Step 2: Where doing it yourself gets painful
Once you go from one feed to a real list, the jobs pile up:
- Finding the feed. People give you
https://wordpress.org/news/, nothttps://wordpress.org/news/feed/. You need to read the page's<link rel="alternate">tags and try paths like/feed,/rss.xmland/atom.xml. - YouTube channels. A channel's feed lives at
youtube.com/feeds/videos.xml?channel_id=..., so you first have to turnyoutube.com/@nameinto a channel ID. - Podcasts. Audio URL, file size, duration (
1:02:03or seconds), episode and season numbers sit in iTunes-specific tags. - Duplicates. The same story appears in two feeds, or with different
utm_tracking parameters on the link. - Only new items. If you run it every morning, you want just the posts you haven't seen, which means storing state between runs.
- Broken feeds. A 404, an HTML page instead of XML or a timeout shouldn't crash the whole batch.
None of this is hard on its own, but together it's a small project you then have to maintain.
Step 3: Convert RSS to JSON with one API call
I built RSS Feed to JSON & Markdown – Atom, Podcasts, YouTube on Apify to do the steps above. You give it feed URLs or plain website URLs, and it returns one clean JSON item per post. You need a free Apify account and your API token from Settings → API & Integrations.
pip install apify-client
import json
from apify_client import ApifyClient
client = ApifyClient("YOUR_APIFY_TOKEN")
run_input = {
"urls": [
"https://hnrss.org/frontpage", # an RSS feed
"https://wordpress.org/news/", # a website: the feed is found for you
"https://github.com/nodejs/node/releases.atom", # an Atom feed
"https://feeds.npr.org/510289/podcast.xml", # a podcast
],
"maxItemsPerFeed": 10,
"sinceDate": "7 days",
"includeContentMarkdown": True,
"dedupeAcrossFeeds": True,
}
run = client.actor("ryumoon-data/rss-feed-converter").call(run_input=run_input)
items = list(client.dataset(run["defaultDatasetId"]).iterate_items())
print(len(items), "rows")
for item in items:
if item.get("isError"):
print("FAILED:", item["inputUrl"], item["errorCode"])
continue
print(item["published"], "|", item["feedTitle"], "|", item["title"])
with open("feed_items.json", "w", encoding="utf-8") as f:
json.dump(items, f, ensure_ascii=False, indent=2)
Every item has the same shape whatever the source format: feedTitle, feedUrl, feedFormat (rss, rss-podcast, rdf, atom, atom-youtube or json-feed), title, link, guid, published and updated as ISO 8601 UTC, author, categories, a plain-text summary and contentMarkdown. Podcast items get a podcast object (durationSeconds, episode, season, explicit) and an enclosure with the audio URL; YouTube items get a youtube object with videoId, videoUrl, thumbnail and views.
A feed that fails doesn't stop the run. It becomes one error row with isError, errorCode (for example NO_FEED_FOUND, HTTP_404 or NOT_A_FEED) and inputUrl, which is why the loop above checks for it.
Here is a real item from the Planet Money podcast feed (from the Actor's README, shortened):
{
"feedTitle": "Planet Money",
"feedFormat": "rss-podcast",
"title": "Inside the insurance industry's destruction lab",
"published": "2026-11-06T22:30:00.000Z",
"enclosure": { "url": "https://.../default.mp3", "type": "audio/mpeg", "length": 31091653 },
"podcast": { "durationSeconds": 1943, "episodeType": "full", "explicit": "no" }
}
The same thing in JavaScript
import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });
const run = await client.actor('ryumoon-data/rss-feed-converter').call({
urls: ['https://www.youtube.com/@mkbhd', 'https://wordpress.org/news/'],
maxItemsPerFeed: 10,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const item of items) {
if (!item.isError) console.log(item.published, item.title, item.link);
}
The YouTube channel URL works as-is: the Actor turns /@name into the channel's video feed, so you don't need a YouTube API key.
Step 4: RSS to CSV
The dataset can be downloaded as CSV or Excel straight from Apify Console, or you can flatten it yourself:
import csv
fields = ["published", "feedTitle", "title", "link", "author", "summary"]
with open("feed_items.csv", "w", newline="", encoding="utf-8") as f:
writer = csv.DictWriter(f, fieldnames=fields, extrasaction="ignore")
writer.writeheader()
for item in items:
if not item.get("isError"):
writer.writerow(item)
Step 5: Get only new posts every day
For monitoring (competitor blogs, news, release notes), turn on Only new items. The Actor remembers what it has already delivered under a memory name you choose, so each run returns only posts you haven't had before:
run_input = {
"urls": ["https://techcrunch.com", "https://wordpress.org/news/"],
"onlyNewItems": True,
"stateKey": "tech-monitor", # use a different name for each monitor
"keywords": ["AI", "startup"], # optional: keep only matching items
}
Save that input as a task in Apify Console and add a schedule (for example every morning at 7:00). You can send the results on to Google Sheets, Slack or Notion with Apify's Zapier, Make or n8n integrations.
If a feed only includes a teaser, fetchFullArticle: True opens each link and adds the full article as Markdown in articleMarkdown. Only use that on sites you are allowed to process.
What it costs
Prices from the Apify Store on 17 Nov 2026: $0.001 per feed item saved (so $1 per 1,000 items) and $0.002 extra per full article, charged only when the article is fetched successfully. There is no start fee. Error rows, duplicates, items removed by your filters and items you already had with Only new items are not charged. Apify's normal platform usage applies on top.
So 20 feeds × 50 items is 1,000 items, or $1.00. A daily monitor that finds about 30 new posts a day comes to about 900 items, or $0.90, a month.
Tips and limits
- A feed only holds recent items, usually the latest 10–100. No tool can get older posts from a feed than the site publishes. Run it on a schedule to build up history from now on.
- Start small. Try two or three URLs with
maxItemsPerFeed: 5and check the output before you add your full list. - Proxies are rarely needed, because feeds are public and meant to be read by software. Add one under Proxy only if a site blocks you.
- Republishing content from someone else's feed may need their permission, so check the site's terms.
Wrapping up
For one feed, feedparser and a few lines of Python are enough. Once you have a list of websites, podcasts and YouTube channels and want consistent JSON with ISO dates, Markdown content, no duplicates and only new posts each day, try RSS Feed to JSON & Markdown with a couple of feeds first, then schedule it once the output looks right.
Comments
Join the discussionNo comments yet. Be the first to comment.