My 2 minute demo video




Twitter rant – How Twitter could be faster and avoid the Fail Whale

  • Twitter
  • Facebook
  • Digg
  • del.icio.us
  • LinkedIn
  • StumbleUpon
  • Blogosphere News
  • email
  • Reddit
  • Technorati
  • Tumblr

Summary:

This is a departure from my normal LinkedIn tips, as I will begin to discuss wider topics in social media.  However, this is just a rant – it’s my personal letter to Twitter to tell them how to speed up their site and stop giving us the Fail Whale.

Details:

Dear Twitter,

You’ve got real problems with performance and reliability, and I can help you fix them, but this is going to take more than 140 characters to explain.

Why I’m qualified

First of all, I’m probably qualified to give advice to the “Big Bird” in Computer Operations at Twitter.  I was the VP of Operations for the search engine that was tied with Google in the year 2000, so I know search and operations really, really well.  We never once had a Fail Whale.

Search takes time

Simply stated, search takes a lot of time and a lot of resources – CPU, memory, disk space, et cetera.  I don’t need to get into all the details, but you should only do a search if someone is going to look at the results.  If you cut down on the unnecessary searches and processing that you do at Twitter, you’d have more resources and time for other things, like

  • people actively looking through tweets
  • people trying to delete DMs (which sometimes takes forever)
  • people doing searches that they’re really going to look at

You are constantly doing unnecessary searches

That’s your problem.

If I do a search through search.twitter.com, and I leave it alone, it will automatically do the search over and over again, apparently every 20 seconds or so.  At the top of the page, it will say

6 more results since you started searching. Refresh to see them.

but you know what?  I’ve gone to the bathroom, left for a meeting, or I’m working in another window.  I’m not planning to return to that Twitter search window for 10 minutes, and you’ll do 30 completely wasted searches to find out how many results I have.

Guess what – when I return, I could just hit the Search button again if I care.  Meanwhile, since so many resources are being used on searches like mine, people using the standard Twitter interface are getting the Fail Whale.  By the way, they no longer think it’s cute.

Tweetdeck

Tweetdeck is cool, but is a ridiculous offender of the problem above.  It is constantly demanding resources for search, more data, et cetera, and no one in their right mind just sits and watches it all day. The customer service people at Comcast might claim they do, but none of them are in their right mind.

Instead of allowing Tweetdeck to update constantly, how about changing the default values so that it only updates every 10 minutes?  Maybe you should force the user to hit a Refresh key to see the most current data when they are really going to look at it.  You may not be able to force the Tweetdeck utility to do this, but you could stop automatically updating it from your end.

Conclusion, and 3 easy solutions

This problem is easy to solve.  If things are getting slow at Twitter, and you’re thinking about putting out the Fail Whale, here are 3 things you could do to improve performance

  1. stop updating searches at search.twitter.com
  2. stop updating the Tweetdeck  applications automatically
  3. stop doing anything else unless someone specifically asked for it with an Enter key or Refresh button

Oh, and if you need a consultant to help you with your operations, don’t waste your time doing unnecessary searches for that either.  I’m right here.  Contact me @617patrick

You’re welcome.

Sincerely,
Pat

P.S.  when you did maintenance during the middle of the day on May 13th, don’t say “I’ll be back in an hour” like my local cleaners would do.  We don’t know when you left, so we don’t know when the hour is up.  Tell us you’ll be back at 1 PM EST.

Be Sociable, Share!

3 comments

1 Brian { 08.27.10 at 12:12 pm }

Well said Patrick, well said indeed.

I don’t know though, I still think the Fail Whale is kind of “cute”…..in an annoying and exasperating kind of way. Sort of like when you’re watching a movie on TV and it goes into a commercial. You start mumbling something about “stupid commercials, why do they have to put commercials in movies…”, and then you notice puppies running all over the screen and you go, “awwwwwwww”. Before you snap to your senses and go back to the “stupid commercials” rant.

2 Chris { 10.20.10 at 12:02 pm }

Your advice to Twitter is that they give up their main selling point (the dynamism of the service) in order to alleviate a few outages? That’s not the best advice I’ve ever heard.

And besides, things have moved on in web service technology in the past decade. A connection from a browser every 20 seconds is not particularly costly. Certainly it’s something Twitter copes with most of the time. And it’s a scalable problem anyway – it can be solved by throwing more hardware at it. That’s a much better solution for Twitter than having people use their service less.

3 Patrick OMalley { 10.20.10 at 12:48 pm }

Chris,

I disagree.

Twitter’s main selling point isn’t that it updates my page whether I’m looking or not. If I’m not looking, and they update my page, they are just wasting computer power. If they have too much computer power, and never run out, that isn’t a problem, but if they are unreliable during the day, they should fix it.

I agree that “A connection from a browser every 20 seconds is not particularly costly”. However, the SEARCH that they have to do IS costly, and that’s what is causing the fail whale.

Also, “throwing more hardware at it” costs money and takes time. My solution is quicker, and costs almost nothing.

Leave a Comment