The role of business analysis on data migration projects

A few people have asked me: “What is the role of a BA on a data migration project?” I’ll try to summarise some thoughts here.

Data migration projects are all about three things:

• decide which data do you need

• find where to extract it from

• define where to put it to

Plus a bit of transformation in the process :) Let’s have a look.

Photo: 0asa

Just like any project or initiative, to be a success your data migration should serve a certain purpose and be governed by a set of requirements elicited from the interested parties.

For those interested in theory, the requirements to migration typically fall under the “transition requirements” category. The BABOK(r) Guide says:

“The transition requirements describe the capabilities that the solution must have and the conditions the solution must meet to facilitate transition from the current state to the future state, but which are not needed once the change is complete. They are differentiated from other requirements types because they are of a temporary nature. Transition requirements address topics such as data conversion, training, and business continuity.”

While training may be out of scope for a migration initiative, data conversation and business continuity are the topics to consider.

BA process for data migration

As a business analyst, most likely you will need to do the following:

1. Identify the purpose of the transition. What business need drives the migration and which outcomes the business tries to achieve. Document the objectives.

2. Based on the objectives, identify which data do you need. Depending on the type of system you work with and intended purpose of that system you may need different data types. It is good to identify the entities you need to migrate (e.g. users, orders, transactions) and separately map any entities that you know you are going to leave behind.

3. For each entity figure out the attributes: what constitutes a single data record. E.g. what do you need to migrate for a user, what do you need for an order, etc.

4. Identify data sources. Where can you pull those data elements from? Which systems? What is the access order for those systems? At this stage you need to understand the format in which the data is stored now, or the format(s) in which the data can be exported.

5. Identify the target systems. Where will the each entity and element be migrated to? E.g. what will be stored in ERP vs. CRM?

6. Define the import format. Which format do you need the data to be in so that it can be imported into the target systems. This will help you inform the data transformation process.

7. Define the transformation process. Think about how do you transform the data from its current format (or the format it can be exported to) to the target format (the one that the target system can understand). Think about the manipulations that need to happen to your data. An example can be that some of the fields will need to be split. Your old system may have one field for Name and Surname, when the new systems has two. Depending on the format of the data, you may suggest splitting the current field into two by the space symbol, if majority of current names are just two words.

Extra considerations

Traceability: Have a think if you need to maintain traceability (because you probably do): which piece of data has been migrated from which system and when. Typically it means you need to add extra attributes, such as batch numbers or labels to help trace the data once migrated.

Quality control: Also, think about quality control approach. After the migration, what will tell you whether the data has been migrated in full and with no error? Typically, the business will have some rules what can help. You may analyse the data to identify outlines and extremes and check the same after the migration (e.g. for an online shop you may learn that all the deliveries in the current data set are no later than 1 week after the order, and all the orders are less than $100, and the total of all orders equals to $1 000 000. So after the migration you can check for the same extremes and borderline values). Extra thing to check is data consistency. If you are migrating interconnected data records, make sure the connections are still respected in the new system.

Business continuity: Finally, consider business continuity. If the data migration is going to take some time (and it probably will), how can you keep the business going without affecting the data sets being migrated. Your strategy can be to use a temporary system to track any data changes that happened during the migration and applying them after the main data set is migrated. Or you may need to double track any changes and record them in two systems. Or you may find your own way.

An important element is understanding of how the downstream systems work. Downstream systems are the systems that are going to consume the data from your new storage as a part of their operations. Because these systems are interacting with the data after the migration, it is easy to miss some vital requirements.

It’s not enough to ensure that the format is good enough for the system you are migrating to, but also how your data will interact with connected systems. For instance, if you are migrating a customer database, it may be central to many other systems, and although your migrated data sits well enough on the new database, it might foul up the next system that picks up the change.

Things become tricky when the migration is not a one off exercise, but a series of multiple system migrations. When this is the case in addition to the above, a good BA will also consider inter-dependencies between the data sources to better schedule the migration between multiple systems and achieve a smoother transition.