The Christchurch Earthquake And Attitudes to Disaster Recovery

I bet that in the wake of Tuesday’s devastating earthquake in Christchurch many organisations are reviewing their Disaster Recovery (DR) and Business Continuity Planning (BCP) situations.

My experience is that most organisations – especially private enterprises – do not give disaster recovery and business continuity enough attention. What underlies this lack of attention are a number of attitudes towards disaster recovery and business continuity planning which hopefully will be challenged by our experiences of the Christchurch earthquake.

From what I have seen, low investment in disaster recovery is usually correlated with poor (or no) Enterprise Risk Management. In my experience risk management is the most appropriate approach to disaster recovery and business continuity, and the enterprise perspective is the best way to look these important parts of the organisation.

Often, when looking at the cost of DR, business units think of return on investment narrowly in terms of increase to revenue. On those terms the extra (often substantial) cost of disaster recovery solutions makes little sense: you won’t make a single extra dollar from investing in DR. If we instead look at our business processes from an enterprise risk perspective, then we examine what the likelihood of a disaster occurring is and what the likely impact of that would be on the whole organisation. For example: there is a risk that a natural disaster would strike in Wellington affecting our data centre which would lead to the destruction or unavailability of key IT systems, this would in turn lead to us being unable to process customer payments which would have a severe impact on cash flow. We can then assess what we might do to mitigate that risk and how much we, as an organisation, are willing to pay to mitigate that risk. In the case of vital business processes and their supporting systems, we may be prepared to pay significant amounts of money to prevent significant system outages (e.g. by buying redundant, geographically diverse systems). In less crucial business processes, we may elect for a lower cost solution, such as a change in business process or accepting some form of delay.

There are a range of other attitudes that underly poor disaster recovery and business continuity practices. One I have encountered (in high-tech industries) is that the only thing needed for business continuity is an IT system disaster recovery plan. This ignores the non-technology aspects of a business that need to be addressed in the case of a disaster (or other business affecting event), such as different business processes, special training needs, or special facilities that may be needed to name but a few. Without this, your beautiful disaster recovery plan may never be initiated, or may be wasted as the organisation is not in a fit state to effectively utilise restored IT systems.

Another attitude that can give rise to poor preparation for a disaster is the belief that disaster recovery involves nothing more than backing up a platform, application, or its data. Obviously if this approach does not include off-site backup then this does not help in the case of even a significant local event, let alone a disaster on the scale of the Christchurch earthquake. In any case a backup is not enough. The additional considerations are numerous: If we backup the data, how do we restore the application? If we backup the application and data, how do we restore the hardware, operating system and database to the right level to get the application to perform? If we backup everything, then how do we obtain and set-up the hardware to be able to use that backup? How do we connect that to the rest of our IT systems and networks? And where would we do that if our facility has been destroyed?

Merely having a disaster recovery plan is not enough, it needs to be tested – as often as is practical. I have personally encountered several cases (and heard of many more) where an organisation’s disaster recovery plan did not work in practice. They thought they had an adequate disaster recovery plan, but when a disaster actually struck it turned out that a key part of the process did not work or was not followed and all of their preparation was useless.

What I am arguing is that disaster recovery has to be more than just keeping some backups. It needs to address how we restore a destroyed system in its entirety – including hardware, operating systems, databases, applications and data – and re-integrate it back into the rest of the organisation (including both people and other IT systems and networks). Investment in disaster recovery needs to be looked at from the perspective of risks to the whole organisation, and it should be part of a larger business continuity plan that addresses those non-IT parts of the organisation. And finally, your resulting plans need to be adequately tested.

What the Christchurch earthquake will have done is to make it very real to people exactly what a devastating effect a natural disaster can have on a business. With any luck some businesses will look at their disaster recovery and business continuity planning, and fix the underlying attitudes that are preventing them from being effective.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: