Skip to content

Cross-account full table copy options for Amazon DynamoDB

There are several recommended options to perform a cross-account full copy of a DynamoDB table.

It seems reasonable to compare these options based on several different properties: setup time (low is better), approach (serverless is better), involved resources (few is better), costs estimate (low is better) and new code possibly required (none is better).

Given the above requirements, one of the best available option seems to be the one based on AWS Backup. This solution requires to enable cross-account backup in AWS Backup, but this particular feature might not be available in a developer's hands (e.g. for security or compliance reasons).

The second best might be this other approach based on Spark jobs handled by Glue. Basically, this requires to create:

  • in the target account:
    • a policy which gives write permission to the target DynamoDB table1;
    • a (cross-account) role with the source account as trusted entity, equipped with the above policy;
  • in the source account:
    • a policy which grants sts:AssumeRole on the above mentioned cross-account role;
    • a role with AWS Glue as principal, equipped with the above policy and read permissions on the source DynamoDB table and S32;
    • a Glue job as described here in the section "For a read and cross-account write across regions", equipped with the above role.

How to handle throttled requests

With this solution, Glue will generate a high rate of write operations, which might led to throttled requests. To avoid DynamoDB write exceeds max retry kind of error, two possible solutions are:

  • increase the number of retries to handle possible throttled DynamoDB write operations to the target table, as suggested here;
  • prepare the target table with a "pre-warm" phase.

For reference, a test made with a 4,5 GB table took about 35 minutes, generating a total cost of 17 USD:

  • 7,4 USD for the execution of the Glue job (source account)
  • 2 USD for the read operations onto the source DynamoDB table (source account)
  • 7,6 USD for the write operations onto the target DynamoDB table (target account)

  1. This table must already exists. ↩

  2. This is needed to let Glue download the job script from S3 assets bucket. ↩