Data Migration

To migrate the data of your tables, you must have your S3 bucket in the same region as your Redshift cluster. Data migration with S3 buckets in regions other than the provided Redshift clusters will be added in the future.

Description

SnowConvert for Redshift migrates the data of your Redshift tables by unloading it to Parquet files in an S3 bucket that you must provide. After the files are created, the application will copy the data directly from those files to the tables deployed in Snowflake using an external stage that must be created before executing the data migration process.

SnowConvert Data Migration Process.

Data Types

The following data types are currently not supported by the data migration feature:

• HLLSKETCH

Prerequisites

Before executing the migration of your data, you need the following prerequisites:

1. Have an S3 bucket in AWS in the same region as your Redshift cluster or Redshift Serverless instances.

2. You need to create an IAM role associated with your Redshift instance that can unload the data of your Redshift tables into your S3 bucket. The IAM role must have the following policy configuration (the following configuration can be used by any database user, to restrict access to this role you can read this guide):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:GetBucketLocation",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::<your_bucket_name>/*",
                "arn:aws:s3:::<your_bucket_name>"
            ]
        }
    ]
}

1. You must have an AWS external stage in Snowflake that references the S3 bucket where the data of your tables will be unloaded. You can follow this guide to setup an S3 storage integration in Snowflake as well as an external stage.

2. Be connected to the Redshift instance and the Snowflake account where the DDL code was deployed.

Execution

1. Click on Set Data Migration Settings to add the following information:

   1. The database, schema, and name of the S3 external stage you previously created.

   2. The IAM Role ARN that will be used to unload data from the tables into Parquet files in the S3 bucket URL of your external stage.

   

   Data Migration Settings Modal.
2. Select the tables for which you want to have their data migrated to Snowflake.

3. Click on Migrate Data, which will start the data migration process by unloading the data into the S3 bucket and then copying the data from those files to the target tables in Snowflake.

4. The data migration column will be updated, indicating if each table had its data migrated successfully or not.

Tables with data migration status.

Results

1. This page validates the number of rows moved from the source tables to the target tables.

2. Each row contains the following information about the migrated table: source schema and table name, target schema and table name, and the number of rows loaded.

3. If you want to execute another data migration process for more tables, you can click on Retry Data Migration.

Data Migration Process Results.