Steps in a Simplified Approach to Data Migration

Steps to Peacock Pond, Florida, Courtesy of Robert Smith, Australia

Steps in a Simplified Approach to Data Migration

This Page describes a series of Steps in a Simplified Approach to Data Migration between Source Systems and Target Systems.
It covers the basics of Extracting data from a number of Sources, Transforming it and Loading it into a Target Database.

Preparation ...

Setting-up the Simplified Data Migration Process ...

Optionally choose a Data Modelling Tool with Reverse Engineering Capability,(such as ERWin).
Optionally define and create the Data Dictionary.

Identify all the required Data Sources, and an 'owner' for each Source. (Data Feeds, Legacy Systems & Operational Data Stores).
If you want to migrate Access Databases to Oracle,MySQL, Sybase or PostgreSQL, then check out MDB Tools, from Brian Burns.
Define the Data Items required, in consultation with the Users.
Optionally create the Data Models for the Source Data.

Define Data Validation Checks (bottom-up, meaning identify available low-level data, such as Invoice Dates,Amounts,etc. ).
Define Validation Rules, (eg Start Date earlier then End Date).
Define Clean-Up Business Rules for Source Data, (eg Set default End Date).
Optionally carry out an Audit of the Data Quality in major Databases, (bottom-up and top-down).
Optionally evaluate the benefits of a Data Cleansing Product.

Define the Staging Area, with 'Source' Tables to store Extract Files, and 'Target' Tables to store data after transformation.
I prefer to have Target Tables that exactly reflect the final Database so that the data can be loaded in directly,
in the knowledge that there is no chance of errors in the final load.
Optionally create the Business Data Model for the Consolidated Database

Define the Data Mapping between Source and Target Data Items.

Optionally create a CRUD Matrix to identify the interactions between Data and Functions.

Define Acceptance Tests for data in the Integrated Database.
Executing the Data Migration Process.
1. Run Acceptance Tests on the Integrated Target Database,(esp. Referential Integrity).

[ Home Page | Ask Me a Question | Email Me | Site Map ]