EBS/RDS DR & Recovery with KMS key

EBS/RDS DR & Recovery with KMS key

When copying cross account an EBS/RDS Volume encrypted with custom KMS,  a KMS key should also be available in the other account.
There are 2 ways that CPM uses for checking KMS key - Alias & Tag
 
KMS Tag
When using custom tag,  you are telling CPM to use that custom ARN for the backup/DR (which could be in same account or even 3rd account).  A recovered resource will be encrypted with it as well.
The flow is:
      1. DR account snapshot is encrypted with custom key X 
      2. Volume recovered (cross-account) with custom key X.
      3. DR and recovery use the same key X, because it’s custom - original volume/instance has this tag (we use instance tag if it has one).
      4. When using a custom key, the original volume’s encryption is not relevant during DR.

This is useful if you have KMS in 3rd account that you want to use for all resources in both accounts.

Alias:
When there is no custom tag, CPM will search in target account/region for KMS with same alias as the original KMS alias and will use that for the DR process.
So in this scenario you just need to make sure KMS’s on both accounts have same alias, for example account A’s KMS key with alias ‘my_kms_1’ & account B’s KMS key with alias ‘ my_kms_1’.
Then when you recover to account A it will use account A KMS.

Possible Errors:

If CPM can't find matching Alias then you will see following error in backup log for EBS DR
Volume DR copy snapshot failed (to DR account). No matching KMS alias on target region (source US East (N. Virginia), target US East (N. Virginia), snapshot snap-01b452ad99d4ce30d, KMS alias: alias/test2)


And you will see similar error for RDS
ERROR:  start_copy_region(dr_rds.py:301) RDS DR copy snapshot failed (in Backup account). No matching KMS alias on target region (source EU (Frankfurt), target Asia Pacific (Singapore), RDS snapshot cpm-policy-RDS-daily, KMS alias: alias/ec2-prod-rds-custom-kms-key.
When copying RDS cross account/region, it will first copy to the same region cross account and then to the target region within the target account ( this is due to AWS limitation that prevents from copying directly), so you will need to make sure that key exist in all location with the relevant alias 
Also in this case, the resulting snapshots will be full and not incremental.

Another error that you might encounter is:
ERROR:  check_region_copy_state(dr_volume.py:409)  Volume DR region: EU (Ireland), snapshot snap-123456789 failed (source snap-987654321), has status error (policy mytestpolicy,to DR account). Error: Given key ID is not accessible
 
As part of copying snapshots cross account, the target account need to have access to the source KMS , otherwise it won't be able to re-encrypt the snapshot.
Go to KMS on the source account and add the DR account under "Other AWS accounts" on the Key Policy and the DR should be successful.

If you get this expection: Exception The source snapshot KMS key [arn:aws:kms:eu-central-1:123456789012:key/fdf5c33f-d9ba-4b1-99e3-3c1d45675b] does not exist, is not enabled or you do not have permissions to access it.