Saving data storage costs with a cloud data migration

Background

A North American logistics company offering transportation services and with more than 75 distribution sites called on Keyrus for a problem related to migration to the cloud. As the company sought to improve its analytics, leverage lucrative opportunities, and reduce costs, it explored the possibility of moving its 3rd party hosted data to the cloud. While the 3rd party was storing the data on a SQL Server database, the hosting cost for it was reaching $180,000/year, partially due to the large amount of data storage required in order to keep the historical data for regulation purposes. As a result, Keyrus was asked to help migrate its data to Snowflake in order to benefit from the low storage cost, while recommending the right technology and process to carry out this SQL Server to Snowflake data migration.

Challenge

- On premise data warehouse is too expensive, inflexible - Cloud migration can easily become complex undertakings = find the right data integration solution, ensure data quality and configure database syncs.

Approach

Considering this was a lift and shift migration of the entire SQL Server database to Snowflake, Keyrus recommended Rivery. With the choice of Rivery technology, the cloud data migration process is automated for every kind of database. Whether handling an on-premise legacy database, or a cloud-based data system, Rivery quickly, efficiently, and smoothly migrates data into the cloud. - On-premise: Migrate any on-premise relational database, including Oracle, MySQL, Microsoft SQL Server, and PostgreSQL, to a cloud data warehouse without inefficiencies and busywork. - Cloud-based: Perform cloud-to-cloud database replication, such as Redshift to Snowflake, and continuously update data changes in real-time. Rivery offers two data replication methods that automate cloud database migrations, no matter the use case. - Multi-Tables Multi-tables automatically migrates a database to the cloud by pulling data directly from tables. o Auto-migrate a database to the cloud using the standard extraction method. o Migrate an entire database using a single data pipeline. o Eliminate complicated configurations and manual grunt work. - Change Data Capture (CDC) Change data capture instantly and automatically syncs a database with your cloud data warehouse, as soon as changes occur. o Always keep source and target databases synced. o Act on real-time data to beat the competition. o Cut time & costs by loading data incrementally. For this migration, since the source database was only storing historical data, a multi-table migration was carried out allowing to move the entire database structure and data to Snowflake in days. This allowed saving time avoiding the need to plan out and execute complex migration processes such as creating destination database structures or adjusting column data types.

Benefits

Two new tools were configured for current use cases and future developments: - Rivery - Snowflake The SQL Server database was lifted and shifted to Snowflake: - Migrated data to Snowflake = $180,000 annual savings by using Snowflake instead of a third-party vendor and leveraging Snowflake’s cheap storage cost - 2.3 billion records, 229 tables and 4 views migrated and validated against source. The conclusion is quick and easy to make: In most cases, cloud data warehouses store, manipulate, and analyze massive amounts of data in a more efficient, scalable, and cost-effective way. Migrating data into such a solution can be a relatively quick and smooth process when using the right set of tools and processes.

Share this key play