Data migration is an integral part of any ERP software upgrade. Whether you’re moving from introductory software like QuickBooks, or legacy ERP, it is important that any significant existing data is properly imported into your new solution. This service is typically included as part of the software implementation process; however, it can vary significantly in terms of scope of work and costs from one vendor to the next. Understanding your options for data migration and the factors that influence cost will help your business decide what software vendor to work with. Even though costs will vary from one vendor to the next, aside from training expenses, the data migration process tends to make up the majority of the implementation costs and therefore your budget should reflect this.
When it comes to data migration options there are typically several available depending on the vendor:
- The vendor will charge you a set rate for moving your data
- The vendor will charge you by the hour for moving your data based on a predetermined estimate of the time required
- The vendor will provide you with a data import tool or spreadsheet for you to manually update and import the data yourself
- You will be solely responsible for manually inputting any data into the new system
- Some combination of the above
For the most part, unless you’re a start-up company, some form of data migration will be required. If you’re a start-up company with no transaction history data, customer information, vendor information or product information, it is usually ok to manually enter this information into the system as needed. Even if you do have some product/vendor/customer information, the volume is low enough that manual data entry is still your best and most economical option. For most other businesses, it is never recommended to move your own data. Even if this seems like a good idea initially as a means to cut implementation costs, it can end up costing you even more money down the road, as you lose productivity due to inaccurate information in the system, or when you run into issues moving the data yourself and end up having to pay the vendor to do it anyways. We recommend you work with a vendor who has experience moving data and will perform the entire data migration process at an hourly rate, as discussed with you beforehand. To better understand the benefits of this method, let’s explore the data migration process itself using Blue Link ERP as an example.
Step (1) Initial Data Analysis
The first step in Blue Link’s data migration process is an initial data analysis and discussion with your team to assess the best approach for migration – what data needs to be moved, where is the data stored, is there an opportunity to clean up existing data etc. At a bare minimum, you want to make sure that your existing data is stored in the correct fields and following the correct formats. This step also provides you with the opportunity to improve the quality of your data. Do you have a lot of duplicate customer records? Are you happy with existing product codes? Do you have duplicate inventory items to account for different UOMs? If you want to change anything, now is the time to do so. For example, if you simply wish to change Product Codes, we can provide you with a spreadsheet template that includes an extracted list of all existing Product Codes and a column for you to include Product Code changes. Changes can be done with the help of a simple spreadsheet template, or, if the data follows a logical pattern, our team can write a data conversion import routine.
As a general rule, it’s usually best to keep the vendor responsible for providing guidance on data types and formats, and for the actual import process, while taking ownership yourself of the quality of the data. Either update data manually in your existing system, get the vendor’s assistance in dumping data into a spreadsheet to review and clean or have them write an import conversion routine to automatically make changes.
Step (2) Accessing the Data
The second step involves our team of knowledgeable consultants working with you and your team to actually access and extract the raw data to begin the migration process. Blue Link does this with the help of its data conversion import routine. This step will vary in complexity depending on your existing company set-up, how many systems you want to extract data from (for example accounting systems, CRM software, warehouse systems and eCommerce platforms) and whether you are moving data for multiple separate entities. Although you can migrate as much or as little data as you want, most often, the data that is transferred includes vendor, customer and product information, historical sales data, A/P and A/R outstanding, open sales orders, GL account numbers and balances and inventory quantities.
During this step, Blue Link will validate the data, identify any issues and update any data changes that follow a specific pattern. For example, does the tax authority associated with the salesperson in the migrated data actually exist in the Blue Link data? Do the terms associated with that salesperson exist in the new Blue Link data? Blue Link’s data conversion import routine will identify any missing data or “orphan records” where data exists in one table but not another. Another example of how the validation process works is if the conversion import routine tries to populate a field in Blue Link with text that is longer than the number of characters for that field. Whatever the issue, the routine will create a record of the missing or incorrect data so that it can be fixed afterwards without stopping the import from continuing.
This step also includes “massaging” or adjusting the data according to specific patterns. For example, if you have different SKUs for each UOM of a specific product but the information is stored following a pattern (SocksPair, SocksDozen, SocksBox) our consultants can work with this pattern to create a single SKU with different UOMs in Blue Link. Different software solutions will store data in different ways and using different fields, titles and structures. Therefore, part of the validation process involves ensuring that the data makes sense and follows structures and functionality in Blue Link.
Step (3) Final Data Extraction
Step three involves re-extracting the data from your existing system right before Go-Live using Blue Link's data conversion import routine to ensure the data is the most up-to-date. Before this happens, however, there are a couple of considerations to keep in mind. Certain information that was originally migrated ahead of Go-Live does not need to be moved again. This includes data such as GL Codes and Salespeople as any new additions can be manually updated (as long as you have a good process for tracking this). Instead of spending the time and effort to move data that does not get updated frequently a second time, we will advise the customer on which data they should manually enter if they add anything new between the initial import and Go-Live. However, current/open transactions and balances will need to be re-migrated at the point of Go-Live, and then with historical data such as sales history, timing is much more flexible – you could import sales history any time after you’re already live on your new system. Alternatively, you can save a local copy of your historical data for reference as opposed to migrating it into Blue link depending on the data in question.
As you can see from the information above, the data migration process is complex and time-consuming. Trying to move the data yourself does not make much sense when it’s not your area of expertise. Charging a set fee for data migration can cause issues before you start to see and play with the data as there are a lot of unknowns. Since there are so many things to consider when it comes to migrating data, be wary of choosing a vendor that charges a set fee for the migration process. It might seem like a smart choice initially but can cause issues and cost money down the road. A better option is to work with a vendor that provides an estimate of how long the work will take and then provides you with an hourly rate, keeping you informed on the progress of the process to avoid any surprises throughout. If they are an experienced software vendor, they will have years of information from past implementations that will make their estimates very accurate. Remember that one of the benefits of using Blue Link ERP is that the system controls data entry. It prevents users from storing and using “bad data” and it provides one database from which to manage all your business information. Having access to accurate data in real-time is key to running a successful business. Spend the time and money to properly move your data and it will help run your business going forward.
The Dangers of Migrating Your Own Data
- If you don’t understand the technology behind the new system, there can be mistakes when trying to move data yourself
- It takes away from your day-to-day responsibilities
- It does not allow for any data clean up
- During this time you will be busy learning how to use the new software, training employees, changing processes etc. and so trying to move data yourself adds a level of complexity and stress that is unnecessary
