They say love is blind, and in the case of systems management tools it seems especially true.  Despite multiple bad previous experiences, we’re going back for more.  In recent days we’ve been implementing Zyrion Traverse for our hosting operations.

If you Google “systems management” or “systems monitoring” you’ll quickly learn that these tools are now essentially commodity software.  There are dozens, if not several hundred reinvented wheels out there to choose from, whether they be free, ad-supported, or pay-for products.  In fact, the space is so full of purple cows at this point that one has trouble judging between the shades.  If you’re thinking of doing a start-up to produce Yet Another Systems Management Tool, differentiating yourself will be the primary challenge.  Writing the software will be only 10% of your effort.

So, with all the eligible prospects out there, how does one select a systems management tool?  All of the worthwhile products have the same set of basic features: discovery & polling, thresholding and alerting, and data collection and reporting. Many of them also sport attractive eye candy in the form of topographical status displays (the practical usefulness of which is often questionable and a subject for another day) and three dimensional displays of node relationships and performance data. Some can do performance base-lining from which they are able to tell when a system begins to deviate from it’s normal performance profile.  More ambitious products attempt to correlate the blizzard of event traffic blowing through the network and do root-cause analysis.  The best ones are highly configurable, extensible, and offer great API’s, because no matter what you product you select, you will invariably be faced with some amount of customization to make it really fit your environment.

We narrowed our search to products that reflected the ideas  guiding the design of our own in-house tool until we will ceased its development several years ago. We were headed to the next level of system managements: provide business views of our operations.  Most tools focus strictly on the status of discrete elements and their resources, e.g. a router is down, a server is out of disk space, etc.  While that’s certainly useful information to a degree, the real question is, what does it mean to my business?   If disk space is low on a test server, how much do I care?  Should a red light on that condition cause a severity 1 ticket to be opened?  Is the downed router affecting my ability to ship product or simply the archiving of my 2009 backups?

We looked for a tool that could take the various elements that make up our major business systems and allow them to be defined under a small set of  dashboard lights, one for each business system, that would help us understand the impact of the various events occurring across the enterprise.  So, for example, if the “Order Entry” light goes red, we know it’s an all-hands-on-deck situation because of the well-known equation: no orders=no revenue.  If on the other hand, the “R&D” light goes red where all of test systems are located, there is no need to call in the cavalry.

Because human resources are not easily cloned in a crisis, the relative business value of a red light is extremely important to know when multiple red lights are on at the same time. Which ones should get the immediate attention of our limited human resources, and in what order?   A business view answers those questions, especially if you happen to know the dollar cost of downtime for each of your business systems.

Being in the hosting business, we also needed a tool that could segregate our customers into their own groups and provide a similar business view capability. This feature, usually referred to as “multi-tenancy,” typically comes with the ability to give each customer a personalized view into the business systems hosted in our facility.

After several months of shaking the trees and examining their fruits, we settled on Traverse. It does the basics well, has multi-tenancy, and supports business views. The company appears to be stable. The team that came on-site to help with implementation was capable, and the price didn’t break the bank. While I’m not endorsing Traverse as the be-all-end-all nirvana gotta-have-it systems management tool, it does meet our requirements, and it’s definitely an option worthy of your evaluation if you happen to be in the market for a new system.

With all this goodness, it would seem like our new fiancé is a keeper, and yet the ghost of Systems Management Tools Past still lingers.  Hopefully this relationship will last. Hmm…maybe we should sign a prenup…

//spk

Tweet This Delicious Digg This Post Stumble This Post

At this time of year my focus changes from IT to LIONEL infrastructure.  It’s time to begin building this year’s Christmas layout!   Ah, the joy of no project plan or business case!   And looking at the calendar, Christmas day is quite visible on the horizon, so I need to get moving.

With Christmas being on a Friday this year, the early days of that week will likely not be the most productive in the IT shop as the staff winds down the year with inter-office daytime soireés , extended “lunch hours” for last-minute shopping, vacation time, and so on.  From a online availability standpoint, this is clearly not the time to be making any major changes to your critical systems.

Here are a few ideas for redeeming the time during the slow days before Christmas Eve – those days when few creatures are stirring in your shop:

1. Check your software patch levels.  Are you up to date on critical systems?  If not, make plans to get current after the holidays.
2. Check you hardware maintenance agreements.  Is all your gear covered? How about the hardware that’s going out of warranty Real Soon Now? Are all you agreements current?
3. Check your software licenses.  Are you using more than you own?  Funny how that creeps up without notice.  You may want to square up with your vendors at year-end fire sale prices rather than wait until January.
4. Have your sysadmin’s check the free disk space across your server farm.  Is it time to order more storage, or simply clean out the dead wood?  If a file hasn’t been referenced in the last 12 months, archive it or ask the file owner if you may simply delete it.
5. Check for unnecessary VM sprawl.   Do you have virtual servers that you can decommission?
6. Review your backup strategy.   Are all of your critical systems included properly?
7. Test your recovery capability.  Try to recovery a file, a database, and perhaps even an entire server from backup.
8. Declare email bankruptcy and ask your users to do the same.  Don’t start 2010 with 2+GB’s of personal email.  Refuse to be part of the highest form of pack rattery and digital waste known to man.
9. Review your Internet bandwidth usage.  Do you need more or can you do with less?  Do you need to have a chat with any abusive power surfers?
10. Review your private bandwidth usage and contracts.  Are you nearing the end of any contracts?  Is it time to start shopping for better rates?

This is not a list of really exciting stuff to be sure, but they are all important, low-risk things you can do in the inevitable pre-Christmas lull to get your shop off to a good start in 2010.

Christmas Movie Review Department

There are many renditions of Dicken’s “A Christmas Carol” to chose from.   Arguably two of the best star Alistar Sim and George C. Scott, respectively. I personally prefer the George version , but you can’t go wrong with either.  You need at least one of these in your collection.

Christmas Train Department

Also, if you don’t have a train under your Christmas tree and you have kids, grand kids, or you just know kids in your neighborhood, you really ought to head to a hobby shop and check out the Lionel starter sets.   This one will look especially fine under your tree:

//spk

Tweet This Delicious Digg This Post Stumble This Post

In his recent book Free, Wired magazine’s Chris Anderson asserts that bandwidth and storage are “too cheap to meter,” which allows YouTube to let us reminisce and chuckle for free:

While Anderson’s assertion may seem to be true from a philosophical standpoint, it’s not yet true an economic truth, or we wouldn’t have stories in USA Today talking about how YouTube is costing Google money. In June, Michael Liedtke authored a piece shedding some harsh light on the reality of “Free”:

Technology consultants RampRate projects YouTube’s operating losses this year at $174.2 million — far below the $470.6 million estimated by Credit Suisse analysts Spencer Wang and Kenneth Sena in an April research report that became a hot topic on Wall Street and the Internet.The big mystery is how much it costs Google to store and distribute the 20 hours of video that are sent to YouTube every minute.

After conferring with industry experts, Wang and Sena concluded Google will spend nearly $380 million on Internet bandwith, computer hardware, software and data centers.

But RampRate — a specialist in managing technology expenses — believes Google will spend about $83 million to provide the same things to YouTube.

The lower expense estimate presumes Google has negotiated money-saving deals with broadband providers and other behind-the-scenes players that play an integral role in moving data through the Internet’s pipes. RampRate also believes Google’s own propriety technology has helped hold down YouTube’s costs, an idea that Pichette endorsed in his Maclean’s interview.

Although it has been cutting costs to cope with the U.S. recession, Google can still afford to subsidize YouTube with the money it makes through its search engine. Google earned $4.2 billion last year and started off this year with a first-quarter profit of $1.4 billion.

“When people run models, they generally use standard industry pricing for bandwidth, storage, but we build everything from scratch,” [Google's] Pichette said at the time. “So we know our cost position but nobody else does.”

[emphasis added]

Does it seem logical that a service based on raw materials “too cheap to meter” should have trouble making money?  It’s certainly not my intention to discredit Anderson’s thesis about Free, on the contrary, I tend to agree with his primary point that dramatically decreasing costs will change not only the technology we use, but the very way we do business.  YouTube, Facebook and Twitter, all of which are Free, have already revolutionized the way we can all go to market for basically zero cost.

My point is more of a practical one. Bandwidth and storage continue to cost less, but they cost something significant enough that we still negotiate price when we purchase. Even Google does, though they won’t tell us how well they do it. We therefore need to be not only mindful of the cost, but of continuing to properly manage the capacity. Abundance tends to breed waste, and sooner or later Parkinson’s Law comes into effect, at which point we find ourselves breaking out the checkbook to buy more even of  the commodities that are seemingly “too cheap to meter.”

It is still best practice to put tools in place to monitor, control, and efficiently utilize bandwidth. The Internet OC-12 I priced just a few weeks ago was still measured in dollars per megabit, not cents.

It is also still best practice to manage storage, because it still costs real money also. What I mean by “manage” in this sense is giving your sysadmins tools and the time to go through the bowels of your storage systems and look stale old data that can either be deleted, archived to removable media, or moved to cheaper online storage. You couldn’t possibly have heard it, but that was the sound of 90,000 syadmin’s heads simultaneously exploding. For many sysadmins this kind of grunt work is simply beyond the pale, but notice I didn’t say it had to be a manual effort. There are great tools out there to help, FolderSizes being a personal favorite on the cheap end. If you have really deep pockets for some serious automation, IBM’s Tivoli Productivity Center for Data is the luxury car bus option.

And if you’re the one that gets stuck doing the storage management, don’t forget to head over to YouTube for an occasional break…while it’s still Free!  You can find some great “how-to” tips on cleaning your hard disk with an acetylene torch.

//spk

Tweet This Delicious Digg This Post Stumble This Post