Whether buying a new server, new cloud storage, a new computer, or a new software program, you’ll want to think carefully about what information will move over to the new system.

The default is to just “keep everything,” but that’s usually a mistake. Just like when you move to a new office or house, moving systems creates a natural opportunity to purge items that are no longer useful to your business. To avoid cluttering up a new system with outdated information, we help clients develop a data migration strategy.

What is Data Migration?

Data migration is the process of moving data from one system to another. That could mean a data storage location, a database or an application. You will undergo a data migration anytime you:

• Replace an existing application
• Buy new computer hardware
• Move to the cloud
• Bring data into a new database or data warehouse
• Build a new business process

The ETL Process

Data migration is a one-time process that moves data from one system to another. It doesn’t matter where the data is housed – on premises or in the cloud. The transfer of data can be as simple as export-import – or significantly more complex. If the records in system A and system B have different fields or a different schema, the data must be manipulated before it can be brought into the new system.

ETL stands for extracting, transforming and loading. In the ETL process, the data is first extracted from the old system, then it is transformed (either manually or with a software program) so that it fits the format of the new application or database. Then the final step is to load it into the new system.

Types of Data Migrations and Their Challenges

Data migration processes happen on three levels – storage, database and application.

Storage migration

When you do a technology refresh, this is the time to identify obsolete data to reduce storage space and make it easier to quickly find what you’re looking for. We use a variety of storage migration tools to make this process fast and simple. However, what we can’t do is tell you what’s important to keep and what can be discarded. You have to decide that. We’ll ask you questions like:   

• How many documents do you have with names like brochure-final4.docx? Do you need to keep the prior versions for any reason?  
• Why you have the same content in more than one location? Is it for cloud backup? Or are you working around security settings in your systems to make the content available to different audiences?   
• What is this obsolete content costing you? For example, video files take up a significant amount of space. While storage is relatively cheap, employee time is not. When you add up time spent dealing with system performance issues, organizing clutter, and looking for the right content, the overhead costs can become significant.  

Database migration

A database migration occurs whenever you change database vendors, upgrade the database software, or move a database to the cloud. In a database migration, IT professionals look for compatibility issues. Is there enough storage? Will performance be impacted? How do we protect records that require privacy or security? Are there integration points that can fail? Often, a database migration or upgrade is done in a testing environment before being moved to production.  

Application migration

An application migration is the most complex type of migration. The challenge is that each application has its own data model, so the data may have to be significantly manipulated before being brought into the new application.  

You want to think about the ramifications of keeping – or discarding – old records and historical data. For example, in an accounting system, you can start with an account opening balance or you can import every transaction. In a CRM system, you may want to purge anyone you haven’t talked to in a year. We help clients figure out the best application migration strategy for their situation.  

• Will you have to keep the old system running in parallel – just in case? Could you transfer the data to a spreadsheet instead? Do you have compliance requirements you must adhere to? 
• What is the value of that data? A company that sells SAT preparation tests to high schoolers has a finite time period to sell their solution. There’s no point in keeping a record of who was in high school 5 years ago. But a repair company may have a 5-10 year lifecycle between visits, and want to keep those records.  
• How much does it cost to keep this data? Some applications, like email marketing software, charge by the record. Crossing from 4999 contacts to 5000 may cause the price to spike. If 2000 of those people haven’t opened your emails in 3 years, you’re better off deleting them in the move to a new application. 
• Will system speed be impacted? How many users need to access that historical data or that legacy software feature? Can the old solution be retired? 

Data Migration Tools

The advantage of using IT professionals for your data migration is that we have access to a variety of tools to make the process faster and easier. If we can’t find a tool out of the box, we can sometimes build a script to make the ETL work easier. It depends on the scope and scale of the migration project, but in all cases, we look for ways to simplify and automate the process wherever possible.  

In many cases, the software vendor has tools we can leverage. Here’s one example of what Microsoft makes available for clients wanting to transfer hosting to the Azure platform: https://datamigration.microsoft.com/ 

Our Approach to the Migration Process

With the implementation of any new software, or development of any new database or application, we’ll discuss how to manage your legacy data and find an approach that best fits your business. In our planning, we’ll work with you to determine whether the data migration is a one-time data transfer process or requires ongoing data integration and management.  

We’ll look for ways to simplify your IT environment, seeing if we can consolidate databases, and make sure employees can access your systems from any device, especially when they’re not working on premises. In today’s environment, mobility and remote access is a requirement for any new system. That’s why many companies are moving to the cloud.  

Let us help you determine your application requirements and build streamlined business processes, so you have a solution that works for you now – and in the future.  Book a free exploratory session today!