Where to start with a data migration

Many organisations underestimate data migrations. There is inherent complexity moving data from an incumbent system to a different one including data integrity, formatting, testing, security and the user experience (UX) of the new system.

In light of the recent TSB data migration failure, its often easy to underestimate how critical it is to have an effective data migration strategy. In this article we have written our top 9 tips for a succesful migration.

At Cyber-Duck we recommend the following principles for data migrations:

1. Strategy — We recommend a strategy for schema and structural changes so no legacy data is made unusable and legacy/new functionality is accounted for.
2. UX — Any changes to the UX and user journeys, however small, should be documented and tested with real users throughout the project lifecycle
3. Automated testing — Test scripts and automated testing should be utilised to robo-test thousands of permutations from login to functional usage of the system.
4. Betas — Taking a lean approach by running alpha/beta testing will ensure that both users and system analysts are satisfied with the new system.
5. Runbooks — Ensure there is a runbook for deployment so all steps of deployment are documented. If there is an error, it must be possible to rollback without any data loss. Escalation paths should be set and assigned to individuals.
6. Testing environment — The testing environment should be identical to the production environment. Often a lot of problems happen when the configuration between the two differs.
7. Application hosting — The hosting and cloud platform require special attention to ensure they are scalable and secure. An engineering team should be monitoring performance 24/7.
8. Communication — There should be clear communication that change is coming. Internal resourcing and support should be geared accordingly and planned for a crisis.
9. Analysis — Intelligent insights and tracking tools can be used to alert analysts of problems both during testing and in the live environment.

Having worked with large institutions like The Bank of England and Mitsubishi Electric it is key to be process driven. In my organisations we have developed process documents, checklists and risk registers to ensure data migration happens smoothly. Through our ISO 9001/27001 certifications we ensure that these processes are audited and continually improved. This helps to provide compliance, security and ensure bullet-proof accountability. By having the right team structure and PMO functions you can also provide escalation paths to remedy issues before they become problems.

Note: This article is also published on: https://www.linkedin.com/pulse/successful-data-migration-9-steps-danny-bluestone/