从 DynamoDB Table 生成 ImportTable 参数

验证：未记录

适用版本: 未记录
验证范围: 尚未登记验证记录。

本文缩写（英文全称与中文含义）

缩写	英文全称／名称	中文含义
ARN	Amazon Resource Name	AWS 资源名称，用于唯一标识资源
AWS	Amazon Web Services	亚马逊云服务
CLI	Command-Line Interface	命令行接口
IAM	Identity and Access Management	身份与访问管理
KMS	Key Management Service	密钥管理服务
LSI	Local Secondary Index	DynamoDB 本地二级索引
PITR	Point-In-Time Recovery	时间点恢复
PK	Primary Key	主键
S3	Simple Storage Service	AWS 对象存储服务
SK	Sort Key	DynamoDB 排序键
TTL	Time To Live	存活时间；具体指缓存有效期或记录过期时间

1. 目标#

从 source DynamoDB table 查询 primary key、GSI、LSI、billing mode 和 throughput，自动生成 Account B 执行 ImportTable 所需的 import-table-parameters.json。

DescribeTable A1
    -> inspect primary key / GSI / LSI
    -> reject native import when LSI must be preserved
    -> remove read-only status fields
    -> replace target table name and optional KMS key
    -> generate import-table-parameters.json

这个文件只描述如何创建 destination DynamoDB table，不包含 S3 bucket 和 prefix。S3 参数在执行 aws dynamodb import-table 时单独提供。

2. 前置条件#

AWS CLI can read the source table
jq is installed
target table name is known
destination table does not already exist
source table has no LSI, or destination does not need to preserve LSI

检查工具：

aws --version
jq --version

示例变量：

SOURCE_PROFILE=account-a
SOURCE_REGION=ap-east-1
SOURCE_TABLE=direct_conversation_messages_dynamo
TARGET_TABLE=direct_conversation_messages_dynamo

Account A 和 Account B 可以在相同 Region 使用相同 table name，因为 DynamoDB table name 只需要在同一 account/Region 内唯一。Account B 中不能已经存在同名 destination table，因为 ImportTable 必须创建新 table。

3. 查询 Table、GSI 和 LSI#

查看完整 table description：

aws dynamodb describe-table --profile "${SOURCE_PROFILE}" --region "${SOURCE_REGION}" --table-name "${SOURCE_TABLE}" --output json

只显示迁移需要的结构：

aws dynamodb describe-table --profile "${SOURCE_PROFILE}" --region "${SOURCE_REGION}" --table-name "${SOURCE_TABLE}" --query 'Table.{
    TableName:TableName,
    AttributeDefinitions:AttributeDefinitions,
    KeySchema:KeySchema,
    BillingMode:BillingModeSummary.BillingMode,
    ProvisionedThroughput:ProvisionedThroughput,
    OnDemandThroughput:OnDemandThroughput,
    GlobalSecondaryIndexes:GlobalSecondaryIndexes,
    LocalSecondaryIndexes:LocalSecondaryIndexes,
    SSEDescription:SSEDescription
  }' --output json

只检查 LSI/GSI：

aws dynamodb describe-table --profile "${SOURCE_PROFILE}" --region "${SOURCE_REGION}" --table-name "${SOURCE_TABLE}" --query 'Table.{LSI:LocalSecondaryIndexes,GSI:GlobalSecondaryIndexes[].{
    IndexName:IndexName,
    KeySchema:KeySchema,
    Projection:Projection
  }}' --output json

简短显示 GSI：

aws dynamodb describe-table --profile "${SOURCE_PROFILE}" --region "${SOURCE_REGION}" --table-name "${SOURCE_TABLE}" --query 'Table.GlobalSecondaryIndexes[].{
    Name:IndexName,
    PK:KeySchema[?KeyType==`HASH`].AttributeName|[0],
    SK:KeySchema[?KeyType==`RANGE`].AttributeName|[0],
    Projection:Projection.ProjectionType,
    IncludedAttributes:Projection.NonKeyAttributes
  }' --output table

AttributeDefinitions And KeySchema#

获取 AttributeDefinitions：

aws dynamodb describe-table --profile "${SOURCE_PROFILE}" --region "${SOURCE_REGION}" --table-name "${SOURCE_TABLE}" --query 'Table.AttributeDefinitions' --output json

AttributeType:
    S = String
    N = Number
    B = Binary

KeyType:
    HASH = partition key
    RANGE = sort key

AttributeType 和 KeyType 没有固定对应关系。String/Number/Binary 都可以作为 HASH 或 RANGE。类型只在顶层 AttributeDefinitions 定义一次；table/GSI KeySchema 只写 AttributeName 和 KeyType。

{
  "AttributeDefinitions": [
    {
      "AttributeName": "operation_conversation_id",
      "AttributeType": "S"
    },
    {
      "AttributeName": "receiver_time",
      "AttributeType": "N"
    }
  ],
  "GlobalSecondaryIndexes": [
    {
      "IndexName": "operation_conversation_id_receiver_time_index",
      "KeySchema": [
        {
          "AttributeName": "operation_conversation_id",
          "KeyType": "HASH"
        },
        {
          "AttributeName": "receiver_time",
          "KeyType": "RANGE"
        }
      ],
      "Projection": {
        "ProjectionType": "ALL"
      }
    }
  ]
}

Estimate Data Size#

DescribeTable returns approximate size/count values, normally refreshed about every six hours:

aws dynamodb describe-table --profile "${SOURCE_PROFILE}" --region "${SOURCE_REGION}" --table-name "${SOURCE_TABLE}" --output json | jq '
  .Table as $table
  | {
      TableName: $table.TableName,
      ItemCount: $table.ItemCount,
      TableSizeGiB: ($table.TableSizeBytes / 1073741824),
      GSI: [
        $table.GlobalSecondaryIndexes[]?
        | {
            Name: .IndexName,
            ItemCount: .ItemCount,
            SizeGiB: (.IndexSizeBytes / 1073741824)
          }
      ]
    }
'

Export to S3 contains base table items, not separate GSI data files. Migration export size is therefore closer to TableSizeBytes; after ImportTable rebuilds GSIs, destination storage approaches base table size plus all IndexSizeBytes.

4. 判断能否使用原生 ImportTable#

Source indexes	Native ImportTable	Action
无 LSI、无 GSI	支持	定义 primary key
无 LSI、有 GSI	支持	定义 primary key 和 `GlobalSecondaryIndexes`
有 LSI、目标不需要 LSI	支持，但目标没有 LSI	应用不能继续执行原 LSI Query
有 LSI、目标必须保留 LSI	不支持	预建带 LSI 的 table，再使用 Scan/BatchWriteItem 或 S3 自定义导入

GSI:
    supported by ImportTable TableCreationParameters
    rebuilt from imported item attributes

LSI:
    not supported by ImportTable TableCreationParameters
    cannot be added after table creation

如果 LocalSecondaryIndexes 为 null 或 []，可以继续生成原生 ImportTable 参数。

5. 自动生成参数文件#

默认生成器保持 source billing mode、primary key、GSI key/projection 和 provisioned capacity。它会删除 IndexArn、IndexStatus、ItemCount 等只读字段，并在发现 LSI 时失败。

目标 table 使用 AWS owned key 时，将 TARGET_KMS_KEY_ARN 保持为空。需要 Account B customer managed KMS key 时，设置完整 key ARN：

TARGET_KMS_KEY_ARN=

# Example for a customer managed key:
# TARGET_KMS_KEY_ARN=arn:aws:kms:ap-east-1:222222222222:key/DESTINATION-KEY-ID

生成 import-table-parameters.json：

set -o pipefail

aws dynamodb describe-table --profile "${SOURCE_PROFILE}" --region "${SOURCE_REGION}" --table-name "${SOURCE_TABLE}" --output json | jq --arg target_table "${TARGET_TABLE}" --arg target_kms_key_arn "${TARGET_KMS_KEY_ARN}" '
      .Table as $source
      | if (($source.LocalSecondaryIndexes // []) | length) > 0 then
          error("source table has LSI; native ImportTable cannot preserve it")
        else
          ($source.BillingModeSummary.BillingMode // "PROVISIONED") as $billing_mode
          | (
              {
                TableName: $target_table,
                AttributeDefinitions: $source.AttributeDefinitions,
                KeySchema: $source.KeySchema,
                BillingMode: $billing_mode
              }
              + (
                  if $target_kms_key_arn != "" then
                    {
                      SSESpecification: {
                        Enabled: true,
                        SSEType: "KMS",
                        KMSMasterKeyId: $target_kms_key_arn
                      }
                    }
                  else
                    {}
                  end
                )
              + (
                  if $billing_mode == "PROVISIONED" then
                    {
                      ProvisionedThroughput: {
                        ReadCapacityUnits: $source.ProvisionedThroughput.ReadCapacityUnits,
                        WriteCapacityUnits: $source.ProvisionedThroughput.WriteCapacityUnits
                      }
                    }
                  elif $source.OnDemandThroughput != null then
                    {OnDemandThroughput: $source.OnDemandThroughput}
                  else
                    {}
                  end
                )
              + (
                  if (($source.GlobalSecondaryIndexes // []) | length) > 0 then
                    {
                      GlobalSecondaryIndexes: [
                        $source.GlobalSecondaryIndexes[]
                        | (
                            {
                              IndexName: .IndexName,
                              KeySchema: .KeySchema,
                              Projection: .Projection
                            }
                            + (
                                if $billing_mode == "PROVISIONED" then
                                  {
                                    ProvisionedThroughput: {
                                      ReadCapacityUnits: .ProvisionedThroughput.ReadCapacityUnits,
                                      WriteCapacityUnits: .ProvisionedThroughput.WriteCapacityUnits
                                    }
                                  }
                                elif .OnDemandThroughput != null then
                                  {OnDemandThroughput: .OnDemandThroughput}
                                else
                                  {}
                                end
                              )
                          )
                      ]
                    }
                  else
                    {}
                  end
                )
            )
        end
    ' > import-table-parameters.json

如果 source 使用 LSI，jq 返回错误，shell pipeline 失败。不要继续使用空文件：

test -s import-table-parameters.json
jq . import-table-parameters.json

6. 当前 Table 的结果#

direct_conversation_messages_dynamo 当前结构：

table:
    PK=conversation_id(S)
    SK=message_id_time(S)
    BillingMode=PAY_PER_REQUEST
    LSI=none

GSI:
    conversation_id_receiver_time_index:
        PK=conversation_id(S), SK=receiver_time(N), Projection=ALL

    receiver_id_receiver_time_index:
        PK=receiver_id(S), SK=receiver_time(N), Projection=ALL

    conversation_id_message_id_index:
        PK=conversation_id(S), SK=message_id(S), Projection=ALL

    operation_conversation_id_receiver_time_index:
        PK=operation_conversation_id(S), SK=receiver_time(N), Projection=ALL

    conversation_id_seq_index:
        PK=conversation_id(S), SK=seq(N), Projection=ALL

判断：

LocalSecondaryIndexes = null
BillingMode = PAY_PER_REQUEST
OnDemandThroughput = null
SSEDescription = null

result:
    native ImportTable is supported
    omit table/GSI ProvisionedThroughput
    omit null OnDemandThroughput
    omit SSESpecification to use the destination default AWS owned key

生成文件应包含：

TableName=direct_conversation_messages_dynamo
AttributeDefinitions=7 key attributes
KeySchema=conversation_id + message_id_time
BillingMode=PAY_PER_REQUEST
GlobalSecondaryIndexes=5 indexes

完整 import-table-parameters.json#

以下是 direct_conversation_messages_dynamo 可用于 Account B 原生 ImportTable 的完整文件：

{
  "TableName": "direct_conversation_messages_dynamo",
  "AttributeDefinitions": [
    {
      "AttributeName": "conversation_id",
      "AttributeType": "S"
    },
    {
      "AttributeName": "message_id",
      "AttributeType": "S"
    },
    {
      "AttributeName": "message_id_time",
      "AttributeType": "S"
    },
    {
      "AttributeName": "operation_conversation_id",
      "AttributeType": "S"
    },
    {
      "AttributeName": "receiver_id",
      "AttributeType": "S"
    },
    {
      "AttributeName": "receiver_time",
      "AttributeType": "N"
    },
    {
      "AttributeName": "seq",
      "AttributeType": "N"
    }
  ],
  "KeySchema": [
    {
      "AttributeName": "conversation_id",
      "KeyType": "HASH"
    },
    {
      "AttributeName": "message_id_time",
      "KeyType": "RANGE"
    }
  ],
  "BillingMode": "PAY_PER_REQUEST",
  "GlobalSecondaryIndexes": [
    {
      "IndexName": "conversation_id_receiver_time_index",
      "KeySchema": [
        {
          "AttributeName": "conversation_id",
          "KeyType": "HASH"
        },
        {
          "AttributeName": "receiver_time",
          "KeyType": "RANGE"
        }
      ],
      "Projection": {
        "ProjectionType": "ALL"
      }
    },
    {
      "IndexName": "receiver_id_receiver_time_index",
      "KeySchema": [
        {
          "AttributeName": "receiver_id",
          "KeyType": "HASH"
        },
        {
          "AttributeName": "receiver_time",
          "KeyType": "RANGE"
        }
      ],
      "Projection": {
        "ProjectionType": "ALL"
      }
    },
    {
      "IndexName": "conversation_id_message_id_index",
      "KeySchema": [
        {
          "AttributeName": "conversation_id",
          "KeyType": "HASH"
        },
        {
          "AttributeName": "message_id",
          "KeyType": "RANGE"
        }
      ],
      "Projection": {
        "ProjectionType": "ALL"
      }
    },
    {
      "IndexName": "operation_conversation_id_receiver_time_index",
      "KeySchema": [
        {
          "AttributeName": "operation_conversation_id",
          "KeyType": "HASH"
        },
        {
          "AttributeName": "receiver_time",
          "KeyType": "RANGE"
        }
      ],
      "Projection": {
        "ProjectionType": "ALL"
      }
    },
    {
      "IndexName": "conversation_id_seq_index",
      "KeySchema": [
        {
          "AttributeName": "conversation_id",
          "KeyType": "HASH"
        },
        {
          "AttributeName": "seq",
          "KeyType": "RANGE"
        }
      ],
      "Projection": {
        "ProjectionType": "ALL"
      }
    }
  ]
}

这份文件有意省略：

ProvisionedThroughput:
    table uses PAY_PER_REQUEST

OnDemandThroughput:
    source value is null, so no maximum throughput override is copied

LocalSecondaryIndexes:
    source value is null

SSESpecification:
    source SSEDescription is null
    destination uses its default AWS owned key when this field is omitted

TableThroughput / IndexStatus / IndexArn / ItemCount:
    read-only description fields, not accepted table creation configuration

如果 Account B 要使用 customer managed KMS key，在完整文件的顶层增加以下字段：

"SSESpecification": {
  "Enabled": true,
  "SSEType": "KMS",
  "KMSMasterKeyId": "arn:aws:kms:ap-east-1:222222222222:key/DESTINATION-KEY-ID"
}

该字段与 TableName、BillingMode、GlobalSecondaryIndexes 同级；注意给前一个顶层字段补逗号。KMS key 必须属于 Account B 和 destination Region。

7. Review 生成结果#

必须检查：

TableName:
    target table name is correct
    destination account does not already contain the table

AttributeDefinitions:
    contains only primary key and GSI key attributes
    conversation_id/message_id/message_id_time/operation_conversation_id/receiver_id are S
    receiver_time/seq are N

KeySchema:
    HASH=conversation_id
    RANGE=message_id_time

GlobalSecondaryIndexes:
    all 5 required indexes exist
    every key schema is correct
    ProjectionType=ALL
    no ProvisionedThroughput for PAY_PER_REQUEST

encryption:
    no SSESpecification means destination default AWS owned key
    customer managed key must belong to Account B and destination Region

检查不应出现的字段：

jq '
  paths
  | map(tostring)
  | join(".")
  | select(test("IndexArn|IndexStatus|ItemCount|TableThroughput|SSEDescription|LocalSecondaryIndexes"))
' import-table-parameters.json

预期没有输出。

8. 与 ImportTable 命令的关系#

import-table-parameters.json 只传给 --table-creation-parameters：

aws dynamodb import-table --profile account-b --region ap-east-1 --s3-bucket-source S3BucketOwner=222222222222,S3Bucket=PLACEHOLDER-BUCKET,S3KeyPrefix=A1/T0/AWSDynamoDB/EXPORT-ID/data/ --input-format DYNAMODB_JSON --input-compression-type GZIP --table-creation-parameters file://import-table-parameters.json

不要提前创建 destination table。ImportTable 根据参数文件创建 table、GSI 和 encryption configuration，然后加载 S3 data files。PITR、TTL、deletion protection、auto scaling、alarms、tags、resource policy 和 application IAM 需要在 import 完成后单独配置。

Generate Import Parameters