We’ve come a long way in managing and governing data centers. In my last post, I described a data center governance assessment model, and I suspect many of you found your data center governance nearly sufficient. However, even if your assessment indicates good posture in most areas, there are certainly some areas for improvement. And, data center governance isn’t like horseshoes and hand grenades: “close enough” isn’t good enough. So, how do you achieve data center governance success? Structured planning and precise change management.
Categorize the Change
Approach change with a risk assessment methodology. You must understand every potential impact to the business and data center operations, before getting approvals and assuring users the change will not negatively affect operations.
Each change carries varying degrees of risk and should have its own risk profile. A risk profile can be extremely complex or extremely simple. Over the years, I’ve narrowed profile components down to four key areas:
1) Cost. Costs to make the change, including:
- Professional services
- Internal resources and cross departmental charges
- Infrastructure costs such as backup systems, test labs and parallel operational environments
2) Impact. The potential deleterious effect on normal operations must be categorized as high, medium or low impact. Unfortunately, I’ve seen even a seemingly unobtrusive change lead to a high impact event: a technician went into a facility to change out a standalone piece of equipment. However, the technician tripped over a cable that was strung across the floor, taking down the PBX trunks. Essentially, due to poor practice, just sending a human into the data center should have been categorized a potentially high impact event.
3) Time. Change windows are a fact of data center life. The longer the change takes, the longer it might take to back it out, potentially overrunning a change window.
4) Technique. Implementing some changes takes a high skill level. Typically, the more complex the change, the higher the risk, and consequently the need for more precise technique.
Eating the Governance Elephant
Coming back to the data center governance assessment model, breaking down all the necessary changes can be intimidating. To address this, break it down and apply the four risk factors to each change. Then, ranking each change gives you a rational list to prioritize actions. Start with the least risk changes and work your way up to the highest risk changes. In my last post, I described a one to five rating system. If you prefer, you can use a simpler one to three, or high, medium and low approach. The key is to be consistent, across systems and time.
To help you along, here are a few starters:
1) Database updates rate low for impact and technique, but could be high for cost and time. One way to mitigate risk is use an annual application owner registration, putting the burden of providing contacts on the application owners.
2) Labeling the cabinets and everything in them is similar risk to database updates in terms of cost, time and technique, but it carries slightly higher potential impact because any time a cabinet door opens, components can inadvertently get touched, bumped, jostled or zapped.
3) Dressing cables rates high for impact, cost, and time and medium for technique. The degree of risk depends on just how much cable plant change is planned: risk increases proportional to the cable plant size. Again, any time a human touches something in the data center, risk goes up.
4) Hardware restack rates high for cost, impact and time. Technique is typically medium, but it could be high. In other words, hardware restack can be a very high-risk change. Because of this, it must be vetted and extremely well planned. It is often the end process in the whole series of governance efforts after you address all the other dependencies.
It’s important to remember governance is a process, not an end game. Developing and implementing a structured program, including precise, logical, defensible, and consistent change management, is critical for data center governance success. And, there is a side benefit: internal customers give strong approval to a structured program. I’ll elaborate on this in a later post.
To get more useful data center management strategies from Nemertes Research download the Q1 2012 Data Center Knowledge Guide to Enterprise Data Centers.