In his recent book Free, Wired magazine’s Chris Anderson asserts that bandwidth and storage are “too cheap to meter,” which allows YouTube to let us reminisce and chuckle for free:

While Anderson’s assertion may seem to be true from a philosophical standpoint, it’s not yet true an economic truth, or we wouldn’t have stories in USA Today talking about how YouTube is costing Google money. In June, Michael Liedtke authored a piece shedding some harsh light on the reality of “Free”:

Technology consultants RampRate projects YouTube’s operating losses this year at $174.2 million — far below the $470.6 million estimated by Credit Suisse analysts Spencer Wang and Kenneth Sena in an April research report that became a hot topic on Wall Street and the Internet.The big mystery is how much it costs Google to store and distribute the 20 hours of video that are sent to YouTube every minute.

After conferring with industry experts, Wang and Sena concluded Google will spend nearly $380 million on Internet bandwith, computer hardware, software and data centers.

But RampRate — a specialist in managing technology expenses — believes Google will spend about $83 million to provide the same things to YouTube.

The lower expense estimate presumes Google has negotiated money-saving deals with broadband providers and other behind-the-scenes players that play an integral role in moving data through the Internet’s pipes. RampRate also believes Google’s own propriety technology has helped hold down YouTube’s costs, an idea that Pichette endorsed in his Maclean’s interview.

Although it has been cutting costs to cope with the U.S. recession, Google can still afford to subsidize YouTube with the money it makes through its search engine. Google earned $4.2 billion last year and started off this year with a first-quarter profit of $1.4 billion.

“When people run models, they generally use standard industry pricing for bandwidth, storage, but we build everything from scratch,” [Google's] Pichette said at the time. “So we know our cost position but nobody else does.”

[emphasis added]

Does it seem logical that a service based on raw materials “too cheap to meter” should have trouble making money?  It’s certainly not my intention to discredit Anderson’s thesis about Free, on the contrary, I tend to agree with his primary point that dramatically decreasing costs will change not only the technology we use, but the very way we do business.  YouTube, Facebook and Twitter, all of which are Free, have already revolutionized the way we can all go to market for basically zero cost.

My point is more of a practical one. Bandwidth and storage continue to cost less, but they cost something significant enough that we still negotiate price when we purchase. Even Google does, though they won’t tell us how well they do it. We therefore need to be not only mindful of the cost, but of continuing to properly manage the capacity. Abundance tends to breed waste, and sooner or later Parkinson’s Law comes into effect, at which point we find ourselves breaking out the checkbook to buy more even of  the commodities that are seemingly “too cheap to meter.”

It is still best practice to put tools in place to monitor, control, and efficiently utilize bandwidth. The Internet OC-12 I priced just a few weeks ago was still measured in dollars per megabit, not cents.

It is also still best practice to manage storage, because it still costs real money also. What I mean by “manage” in this sense is giving your sysadmins tools and the time to go through the bowels of your storage systems and look stale old data that can either be deleted, archived to removable media, or moved to cheaper online storage. You couldn’t possibly have heard it, but that was the sound of 90,000 syadmin’s heads simultaneously exploding. For many sysadmins this kind of grunt work is simply beyond the pale, but notice I didn’t say it had to be a manual effort. There are great tools out there to help, FolderSizes being a personal favorite on the cheap end. If you have really deep pockets for some serious automation, IBM’s Tivoli Productivity Center for Data is the luxury car bus option.

And if you’re the one that gets stuck doing the storage management, don’t forget to head over to YouTube for an occasional break…while it’s still Free!  You can find some great “how-to” tips on cleaning your hard disk with an acetylene torch.

//spk

Tweet This Delicious Digg This Post Stumble This Post

Remember classic puzzles like “What’s wrong with this picture?” from Highlights Magazine?  Tell me, what’s wrong with this picture:

Yes that’s right, clowns and coffee don’t go together, particularly this clown. We seem to know that instinctively, yet the blogosphere reports that we are apparently ignoring sound instinct in the name of price. The Clown is cheaper than Starbucks, and so we are enticed away from something of known, predictable quality to something, shall we say, less predictable.

McTreachery

Despite being a loyal Starbucks customer, last week I too swung by the local golden arches for a McLatte. Here’s how it went down at the drive-up’s McSpeaker box (imagine Charlie Brown’s teacher’s voice as you read the McSpeaker’s part):

McSpeaker:  Hello, may I take your order?

Me:  Yes, I’d like a medium cafe latte with skim milk.

McSpeaker:  Would you like whole or non-fat milk?

Me: Non-fat please.

McSpeaker: Hot or Cold?

Me: Hot.

McSpeaker: That will be $2.39.  Please pull around.

So far so good, despite my deliberate avoidance of the word “McLatte” or my faux pas of asking for skim rather than non-fat McMilk. I pull around to window #1, pay and receive my change, and proceed to window #2.  At window #2 I wait an eternity for the window to open.  Eventually a young, ponderously pierced McDude opens the window and presents me with a see-through plastic cup holding a milky substance with ice cubes in it. A new dialogue ensues:

Me: I’m sorry, this should have been made as a hot latte.  (said very politely)

McDude: Uh…really?

Me: Yes.   (The petulant McDude now checks the overhead order display for confirmation.)

McDude: Uh…OK…one minute.

As I settle in for another eternal wait, my server surprisingly appears in less than 20 seconds with the proper looking drink: a brown paper McCup with a black plastic McLid. I drive away, yet something seems wrong. The cup should be warm even though there’s a McSleeve to prevent me from suing them over a burned hand.  One sip, and I realized I’ve just been had. The devious McDude has poured the cold latte into a different cup and simply ditched the ice cubes. Surely there must be something in the Geneva Conventions about messing with a person’s morning caffeine fix. Where is Jackie Chiles when you need him?

So, disgusted but realizing I should have known better, I pointed my truck toward Starbucks. I paid a little more, but got exactly what I asked for with exactly the quality I was expecting.

The McLesson

Clearly, coffee is not Ronnie Mac’s forté. This was not my first failed attempt at getting a fancy coffee McDrink though I’ve tried on multiple occasions. Each and every time they have either botched it badly or been visibly irritated to have to break their burger making ritual in order to do obeisance at the latte machine. They pretty much bat 1.000 on the burgers though, because that’s what they’re really good at.

The lesson?  Stick to what you do well and let the rest to somebody else, because you can’t do everything well.  Not even if you have the deep pockets of Ronnie Mac.  Industry type doesn’t matter either, as history shows us that Novell made this same mistake in 1994 when they bought WordPerfect. They strayed away from what they did best at the time (file and print sharing), got into applications, and everything went downhill from there. Their slow descent into mediocrity is well chronicled, but it all started when they took their eyes off of what they were really good at. Tragically, they are not alone.

R.I.P. Wordperfect.  We knew thee well.

McApplication

There are only so many things you or your business can do with excellence – probably less than four – and usually only one in which you can truly excel. If, like the vast majority of businesses today, yours is a consumer of IT infrastructure rather than a provider, it will quite naturally be difficult and more expensive for you to try to deliver IT as effectively as a professional provider can. There is simply too much to know. It may even be a major annoyance like the McLatte machine. Why? Because it’s not your sweet spot. Your strengths will suffer while you’re focusing on things best delegated to others.

Servers, storage, networking infrastructure continue to evolve into increasingly more complex creatures. Unless IT is the sweet spot of your business, it doesn’t make sense to try to keep chasing infrastructure on your own, for much the same reasons you don’t keep factory-trained mechanics on staff to fix company cars. It’s too much, too costly and certainly not worth it.  IT is no different.  You absolutely need to have technology in your business, but there is no reason to bear the burden of it yourself.

Like Starbucks, professional IT providers may cost a fistful of dollars or just a few dollars more, but the results are reliable, predictable, and therefore very much worth it because it frees you to focus on what you do best.  Just make sure your IT provider doesn’t start selling coffee.

Why the movie references to Fistful of Dollars and A Few Dollars More? Well it turns out that Clint Eastwood’s most famous line came after a bad cup of Joe. If gun violence offends you, please don’t go here, otherwise…take a three minute action coffee break with Dirty Harry.

//spk

Tweet This Delicious Digg This Post Stumble This Post

Meet Patches. Keep him healthy and he’ll be with you a long time. Look at that face! Knowing the anxiety Patches suffers when going to the vet, do you religiously take him every time there’s a new medicine on the market, just in case he might catch some exotic bug he has .001% chance of contracting?  No, most likely not.  But you do take him to the vet for regular shots to prevent things dogs of his kind are likely to have problems with – a regular maintenance visit, you might say.

Why it is worth the cost and commotion of going for the maintenance visit, but not every time a new vaccine or pill is announced?  Because the cost/benefit equation is right for one and not the other.

A lot can be learned from Patches about the discipline of patching servers. We are occasionally asked “How often should I patch my servers?” and we get into discussions with a wide variety of customers with widely differing views on the subject. Often though, we find that it largely boils down to one’s view of the world – is your glass half empty or half full? Certainly, we need to keep systems patched to at least the minimum level supported by our software vendors, but given the cost and commotion (dare I say trauma) of the patching process, how far beyond that is necessary or prudent? If you have Internet facing assets, then clearly you want to keep those up to date with the latest security patches as soon as they’re available. But if you have private, stable, non-web assets well behind well-managed firewalls, a less rigorous approach is reasonable. There is no need or rational justification to blindly apply a patch willy-nilly simply because it’s available. Who has not been the victim of downtime because an ill-behaved patch did something that it was not supposed to do? And, lest we forget, rebooting a Windows server after patching is not always a trivial event – just ask the sysadmin of a Blackberry Enterprise server.

Remember the purpose of infrastructure is to keep running – the very namesake of this blog.  Our infrastructure does us no good when it’s down. Every patch brings with it the some level of risk to uptime. So, the obvious thing to do would be to test every patch before we apply it to a product system. Do you? Really? Every time? Or is easier to just apply the latest raft of fixes from say, Microsoft, and just hope for the best? For those of us who have to endure the regular water-boarding process of a SAS 70 Type II audit, hope is not a strategy. Not only do we have to test every patch before applying it to a live system, but we also have to prove that we did so, and that we have a defined process that meets the muster of the auditors.

This process of patching is costly in terms of time, money, and risk. So how often should we patch? Somewhere between hope and SAS 70 lies the right answer for most of us. Like maintaining your car, regular maintenance of a server is necessary to keep a system “on the road.”  This many mean spending time regularly (like an oil change) researching patches to see which of them you really need as opposed to those that make you feel warm and fuzzy, and then testing appropriately first. On the other hand, regular maintenance may not imply regular patching. If a system needs to be running the latest Windows server OS, or the application vendor forces your hand, then you will certainly be patching more often. If on the other hand, you have a functionally stable system that doesn’t change much, has been running well and isn’t the flagship of your ecommerce empire, then you will probably patch extremely infrequently if ever, and that’s OK. We’ve got a Red Hat 7.2 system here that sees heavy daily usage, has not been patched in years, and has not been hacked or had any problems over that same span of time. Sacrilege? Perhaps, but we believe it’s prudence. It could also be Pennsylvania Dutch stubbornness.

You do need a patching process, but it should reflect you particular situation and account for the nature of each of your servers. Like the Pirate Code, best practices in this arena are more like guidelines. You can spend a lot of money and create a lot of headaches with a one-size-fits-all approach.  A socialist patching approach sounds good on paper, but as you would expect with anything socialistic, it tends not to work out well in reality.

Weigh the risks and cost of downtime vs. the potential benefit of a patch.  Part of your process should include a justification phase where IT and business stakeholders have an opportunity to understand what is being patched, why it’s been deemed necessary, and what the possible ramifications are if things go awry. And, most importantly, the stakeholders should have both veto power and the power to determine the scheduling of patch activity.

Patching is a necessary evil, but it is manageable if you take the time to think through the process and come up with a practical plan that fits your business. Or, you could simply delegate the process to folks who know how to both open and close Pandora’s box.

//spk

Tweet This Delicious Digg This Post Stumble This Post