ARTS & FARCES internet

If you’re a regular here, you’ve probably noticed that it was virtually impossible to get into the website for the last couple of days.

Here’s what happened.

The website got slashdotted, blogdexed, and daypopped all at roughly the same time and Zope fell over within minutes. The website was getting so many requests that it would give up the ghost within minutes of each restart. I finally gave up and left the server alone until this morning. Luckily, the pages everyone was looking for were cached on google (just do a search for “farces fun with spam.” This is a good tip to file away for future use when small sites like this get slashdotted.

This website is dynamic—each page is generated on the fly when requested—and only certain page elements (graphic images, header, footer, etc.) can be cached. This makes any dynamic site slower than a static site. There are things I can do to make the site quicker to respond to requests, but their either expensive or beyond my technical ability. And since this site rarely gets slashdotted, it’s a waste of resources.

The good news is that it got us a good deal of publicity and that should result, hopefully, in some underwriting activity. That, in turn, will allow us to produce better content.

The better news is that the bit storm has subsided, and server demand is approaching normal. I'll be analyzing the "slashdot effect" in a future article, but here are some interesting bits in the meantime.

On an average day with no new pieces published, this server receives about 1,500 - 2,000 pageviews. On a day with at least one new article published, the pageview rate goes up to about 5,000 or so. That's respectable, certainly enough to warrant some pretty serious underwriting, but still nothing to really write home about.

The exception, of course, is when one of our articles gets picked up and referenced heavily by other sites. When major sites like Slashdot, blogdex, and daypop reference it, the result is a bit storm: as of about 8:00PM this server has served more than 120,000 pageviews. Now that's something to write home about.

Last Friday I published More fun with spammers, the article that started the bit storm. Here's a breakdown of pageviews for the next four days:

Here's a graphical representation of that data:

That's impressive, but it doesn't indicate anything about all the connections that were refused.