Wai T. Lam is Co-founder and CTO of Cirrus Data Solutions.
When it comes to data migration, most of the advice you read from the “experts” sounds like it came from a fortune cookie you might receive at a restaurant run by Confucius himself.
It’s easy to give obvious advice when talking about data migration, such as “find the right people who can be trusted,” or “avoid downtime and minimize risks,” or “find the right tool that can do everything centrally and automatically.” However, concrete and specific suggestions are probably more helpful. Confucius was a sagacious man, but there are no records of him ever performing a data migration. Therefore, we’re on our own to discover the wisdom of the ages.
When analyzing enterprise-level migration projects, the success stories share some common characteristics. Most of them can be summed up in the answers to the following questions:
What Type of Data is Being Migrated?
It is very important to determine the type of data being migrated since there are disparate migration types. They are:
- NAS-based – network shares are migrated from one system to another.
- LUN-based – disks (LUNs) are migrated from one storage to another at block level.
- VM-based – the entire virtual hosts are migrated from one hypervisor to another.
- Application-specific – one set of database or objects, is migrated from one host to another.
Also, what type of overall migration is this?
- Local migration – within the same data center.
- Remote migration – from one data center to another.
- Cloud migration – to cloud provider (e.g., AWS, SoftLayer, Azure, Google Cloud).
What is the Current Storage Environment?
It is equally important to have a reasonable understanding of the current storage environment. Using SAN as an example, what hosts – in terms of OS and applications – are involved? What are the FC switches and RAIDs? Most of all, how large and inter-dependent is the labyrinth of the SAN? Operational complexity could be exponential and not linear relative to the size of the system. An intimate understanding of the entire picture avoids multiplication of complexity and obviates disasters.
How Much Data is Being Migrated?
It cannot be stressed enough that an accurate and clear estimation of the amount of data must be made. Furthermore, a thorough accounting of the number of hosts, LUNs, and storage systems must be performed. Often, the required effort cannot be judged merely by the number of terabytes that need migrating. Even knowing the total number of LUNs is only half of the story since the number of hosts may be the biggest factor in determining the time and effort required for cutover after the data is migrated. Of course, the amount of data and data rate will determine the absolute time needed for migration, but there are other factors involved as well.
How Much Time Overall Will be Allocated to the Project?
This is almost obvious, but it may not be that simple a question. The amount of data is usually predetermined, so the only other parameter is the data migration rate. This again highlights the importance of understanding the data, application, and environment because knowing if the data can be migrated in a divide-and-conquer manner helps to formulate the strategy – not only to meet the timeframe, but also to control operational complexity, as well as potential cost.
How Active is the Data and How Much Downtime is Available?
How active the data is determines if the migration can be completed in time, or at all. This immediately raises the question: How much impact to production will be tolerated during the migration? Furthermore, during cutover, how much production downtime will be allowed? Knowing this ahead of time allows agreements and strategies to be worked out well before migration starts, and also helps to ameliorate stress during cutover.
What is the Right Tool for the Migration?
The selection of a data migration tool can make or break the project, but what are the evaluation criteria specifically? Again, using the SAN migration as example, a migration tool should be able to:
- Minimize the need to look up every WWPN, host, storage target, and GUID of every LUN, thereby saving hundreds of hours of painstaking and error-prone work.
- Install seamlessly with no disruption to the production environment, which significantly reduces risk during preparation and execution of the migration.
- Provide a clear and intuitive display for the automatically discovered SAN configuration, complete with identities, status, and I/O statistics for all hosts and LUNs to be migrated (as well as those not to be touched), so the migration operation can be organized correctly and accurately with high confidence.
- Intelligently and flexibly allow minimizing, or even eliminating impact to production during migration by, for example, migrating at non-production hours, or automatically yielding to production traffic, with adjustable intensity/throttling.
- Monitor, report, and alert on the migration operation in a centralized manner, while also including the production traffic and activities in general with valuable status and statistics.
- Perform data integrity checking of the intermediate images before cutover occurs, allowing any issues to be discovered before production is switched over to the new storage.
- Allow scrubbing of the data disks after the data is successfully migrated out, especially when storage is to be returned after lease. The scrub level should be variable to allow an efficient mode for speed, or a secure mode with multi-pass scrub to ensure data is securely erased.
- Provide a clear, precise and complete post-migration report detailing exactly what happened during the migration process every step of the way, down to individual LUN activities.
- Demonstrate a solid track record of success and satisfied customers for the given type of migration (through case studies and reference accounts, for example).
Who are the Right People for the Job?
Now we circle right back to the Confucius wisdom: Above all, find the right people – whether using internal personnel or external professional services partners – who can be trusted and who have experience with the given type of migration. How do you know if they are truly qualified? Well, it is actually very simple: Do they articulate the questions posed above with concrete answers, or do they just give you a bunch of fortune cookies?
Industry Perspectives is a content channel at Data Center Knowledge highlighting thought leadership in the data center arena. See our guidelines and submission process for information on participating. View previously published Industry Perspectives in our Knowledge Library.
