April 15, 2003
RSS feeds from Hell
There's a trend out there for creating 'full content' RSS feeds. I think it's bogus.
I greatly value RSS's ability to express lightweight headline-like data from websites. I can skim those headlines quickly then pick and choose which actual web pages I want to read. I get fast overviews from the sites, using very little bandwidth, and the sites get to see if/when I actually come visit them. It's nice deal all around.
If sites can't make me decent headlines with inviting excerpts I don't read them.
Apparently some folks don't ever want to read your website. They'd prefer to read everything inside an RSS reader. I can understand why they'd want this. For people stuck behind slow links, especially ones with only one phone line, having to download the RSS feed and then download later pages does seem like extra effort. Having it all delivered in one full-content RSS feed seems like a cool idea.
Cool until you see feeds like these (randomly selected from Syndic8):
- http://www.syndic8.com/feedinfo.php?FeedID=18262&Section=xml
- http://www.syndic8.com/feedinfo.php?FeedID=26273&Section=xml
- http://www.syndic8.com/feedinfo.php?FeedID=14135&Section=xml
These feeds are each over 250k in size! Where's the lightweight aspect?
To make matters worse, they don't even support gzip compression! Check for yourself using leknor. For pity's sake folks, if you're going to make this huge-ass feeds at least have the common sense and decency to use compression. If your aggregator, like Radio, doesn't support downloading using compression, switch to one that does!
These three feeds, picked at random from more than 150 that were over 64k in size, are an example of tremendous wastes of resources. That they've jammed all that into a feed means anything pulling it aggregator-style is going to get a mountain of data over and over again. Most aggregators in use continue to fail to honor or even check various header schemes.
Why all this waste?
Fortunately, out of over 14,000 working feeds, the numbers of abberant feeds is small. Let's keep it that way, shall we?
For folks that aren't sure of whether it matters, ask yourselves, does it matter if you see hits from your web server logs as people read the pages? Do you depend on that log analysis? A full content feed means you'll never get the same logging data. You'll never know which items were of interest. Since the reader got it all from the full-content feed they have no reason to ever deign to vist your site.
Gee, that's not very polite, now is it?
My point is that while it might seem nice to be able to read the content without ever visiting the web page it's not without it's costs.
For people that want to offer full-content feeds, by all means go right ahead. But keep a close eye on the bandwidth that's wasted serving them up. Your ISP may hit you up with a huge bill should your feed become popular and tons of people using crappy aggregators start picking it up. That and your logs won't show what articles were interesting to the readers.
I'm not sure the convenience of full-content feeds does anything to help the people making the content. It seems like a rather one-sided argument, soley in favor of the readers. I'm all for supporting readerships but I'm not sure others are quite so magnanimous.
I lurve full content feeds. Bill, can you do a special full content feed (with comments) just for me :-)
Posted by: Victor on April 16, 2003 05:14 AMPeople without broadband should be put out of their misery. Seriously though, I MUCH prefer full content feeds, and usually unsubscribe from a feed that is highly abbreviated. If you use something like NewsMonster, it makes a big difference. Perhaps the best solution is to offer both an abbreviated and full feed.
Posted by: Dan Martin on April 17, 2003 11:14 AMThe points you make about bandwidth usage are all correct, and I think there are a few people allready working on schemes for proxy-ing and stuff.
But just hoping that everybody (including all the people doing their weblog for free in their spare time, and thats like 90 % of all weblogs as far as I know) gives their posts intelligent headlines is no solution.
Just look at your own feed, and at this posting right here. You cant deduce that "RSS Feeds from Hell" is about full content RSS feeds increasing bandwidth. It could also have been about syntacticaly incorrcet feeds or RSS feeds from satanistic websites. How about "Full Content Feeds with bandwith costs straight out of Hell" ?
The descriptions given in your feed are the first few words of the posting. This approach works because you mention the issue in the first phrase.
But the obvious way, to get a lean and smart feed would have been to construct a small blurb of the contents and put that in the "excerpt" field when posting to your site.
So to summarize: a lean and informative feed has to contain:
1.) long and unambigous headlines
2.) informative excerpts
3.) and links to the real content
And thats a lot of work to do.
Posted by: Benjamin Heitmann on April 20, 2003 02:23 AMIf you prefer skimming headlines/excerpts in full-post feeds, but your news aggregator doesn't support that, switch to one that does.
My feed has full posts in content:encoded, but it's gzipped, and only includes the last 3 posts (I usually post no more than once a day).
Other options: how about including full-text on only the most recent posts, then dropping down to plain-text excerpts for older posts? Easy to do in Movable Type with [MTEntries lastn="5"] ... [MTEntries lastn="10" offset="5"] (just for instance).
Posted by: Mark on June 11, 2003 06:55 PMI'm not suggesting that seeing the content is the problem. I'm suggesting that the network bandwidth wasted is the problem. Better to have items with 'interesting' descriptions than to get tons of overly formatted crap thrown at me.
The trick on using 'proportional' delivery of content is a good one. Using 'full content' just for newer items would probably work for many feeds.
I'm still a big fan of lightweight feeds. If only because it asks the content provider to put some actual EFFORT into making descriptions that will invite my further inquiry. I'm more inclined to make regular use of a feed when it's giving me inviting titles and descriptions. I've not got the time to read every item. So items with good descriptions generally get noticed much quicker.
Posted by: Bill Kearney on June 11, 2003 07:01 PM
