Moving a data lake from Hive to Apache Iceberg used to take weeks of engineering: a cluster to rewrite the data, type conversions and a rollback plan. Since November 4, 2026, Amazon S3 Tables does it as a managed API operation, with no compute to provision and no need to stop writes to the source table, except for a few minutes at cutover. You pay only for the data copied; the Hive-to-Iceberg conversion itself has no separate charge.
What changes and who it's for
Migration works with external Hive tables registered in the AWS Glue Data Catalog and stored as Parquet in general purpose S3 buckets. S3 Tables creates a new Iceberg table in a table bucket, converts Hive types to Iceberg types, recompresses the data and compacts small files. The source table is left untouched and stays available for reads and writes throughout.
The workflow has three steps:
- Start creates the destination table (read-only) and copies the data. Before copying anything, it writes compatibility reports for the table and for each object.
- Sync copies only what changed since the last copy, even within a partition. You can run it as often as you like, and it also resumes a failed migration.
- Complete makes the table read-write and turns on automatic maintenance. It can't be undone.
It's available in ten regions, including Frankfurt, Ireland and Stockholm.
How to run it
In the console: on the destination table bucket, Create table, then Migrate existing table. From the CLI, using the commands in the documentation:
# Start, and pause once the compatibility reports are written
aws s3tables start-table-migration \
--role-arn arn:aws:iam::111122223333:role/S3TablesMigrationRole \
--destination-table '{"tableBucketARN":"arn:aws:s3tables:eu-west-1:111122223333:bucket/lake","namespace":"sales","name":"orders"}' \
--source-type glueTable \
--source '{"glueTable":{"catalogId":"111122223333","databaseName":"sales","tableName":"orders"}}' \
--migration-output '{"bucket":"arn:aws:s3:::lake-reports","prefixes":{"tableCompatibilityReport":"migration","objectCompatibilityReport":"migration","objectMigrationReport":"migration"}}' \
--review-migration-plan
# Check the state and the reports, then approve the copy
aws s3tables get-table-migration --migration-id <migration-id>
aws s3tables update-table-migration --migration-id <migration-id> --requested-state queued
# Copy changes before cutover, then complete (irreversible)
aws s3tables sync-table-migration \
--role-arn arn:aws:iam::111122223333:role/S3TablesMigrationRole \
--table-arn arn:aws:s3tables:eu-west-1:111122223333:bucket/lake/table/<table-id> \
--migration-output '{"bucket":"arn:aws:s3:::lake-reports","prefixes":{"tableCompatibilityReport":"migration","objectCompatibilityReport":"migration","objectMigrationReport":"migration"}}'
aws s3tables complete-table-migration \
--table-arn arn:aws:s3tables:eu-west-1:111122223333:bucket/lake/table/<table-id>The role needs to read Glue metadata and the source objects, write to the destination table and the report bucket, and trust migration.s3tables.amazonaws.com. With SSE-KMS on the destination, the key must also allow maintenance.s3tables.amazonaws.com.
My advice is to always use --review-migration-plan the first time: the migration waits in pendingReview until you approve it, so you can read the reports before paying to copy a single byte.
The migration policy is set once
Two settings decide what happens to data that doesn't fit, and they are fixed at start: unmatchedSourceFieldsPolicy (source columns missing from the destination schema: fail or drop) and incompatibleValuesPolicy (values that can't be converted: fail or setNull). Both default to failing. With setNull, a file that stores quantity as bigint in an int column migrates with that column set to null for that object: convenient, but silent data loss if nobody reads the object report.
Gotchas and edge cases
- Formats. Parquet only (uncompressed, Snappy, gzip, Zstandard, Brotli or LZ4). No ORC, Avro, JSON or LZO.
- Excluded tables. No transactional (ACID) Hive tables, no tables registered with Lake Formation, none already in an open table format, none that skip header lines.
- Same account and region for source, destination and reports. No federated catalogs.
- Overwrites. A sync fails any object overwritten since the previous copy, even with identical content and even in a versioned bucket. If your ETL rewrites whole partitions, plan for it.
- Partitions outside the table location, or tables using Athena partition projection: move or register them first (
MSCK REPAIR TABLE). - Not copied: Glue table parameters, SerDe settings, column statistics and comments.
- Archived objects in Glacier or the Intelligent-Tiering archive tiers must be restored first.
Compared with the old way
Until now, AWS's reference path was a CloudFormation guidance that spins up an Amazon EMR cluster, runs a Spark Create Table As Select orchestrated by Step Functions, and tears the cluster down afterwards. It works, but it's infrastructure to maintain, a full CTAS copy, and a cluster to pay for while it runs. Managed migration swaps that cluster for an API with upfront reports and repeatable syncs, in exchange for the restrictions above. If your tables are Parquet in Glue, in the same account and outside Lake Formation, there's no longer a reason to build the pipeline yourself; otherwise, the EMR route is still the one that works.
Sources
- Amazon S3 Tables now offer managed migration of Apache Hive tables to Apache Iceberg — AWS News Blog
- Running a table migration — AWS Documentation
- How to migrate a table to S3 Tables — AWS Documentation
- Setting up table migration — AWS Documentation
- Guidance for Migrating Tabular Data from Amazon S3 to S3 Tables — AWS Solutions