RDS Backup or DR fails with Reason: cannot create more than 100 manual snapshots
Issue:
Backup or DR may fail with the following errors:
ERROR: run_snapshots(.\agent.py:880) (database: "dbname") snapshot failed to start. Reason: cannot create more than 100 manual snapshots
ERROR: backup_function_internal(.\agent.py:1204) policy: "policyname". All Snapshots in policy failed!
This is happening as the number of snapshots reached the AWS limit
Solution:
Issue 1:
This could happened if the retention is set to keep to many snapshots or some of the 100 RDS snapshots may be present in other CPM policies or created outside CPM - please check the "Snapshots" view in the RDS Dashboard.
As a temporary workaround you can contact AWS support and ask to raise the limit.
You should either decrease the retention and run Cleanup, or ask AWS to increase this limit:
Please note: decreasing retention will only affect new backups from that point on, so you probably will need to delete some manually (or ask AWS to increase the limit) before you'll be able to take new snapshots.
Issue 2:
It could be that too many snapshots are accumlating due to cleanup issues.
As a workaround you can manually delete older backup records that should have been deleted to free up space - see screenshot below on how to delete backup record from UI