Cleanup may not delete snapshots older than the defined retention for the policy
CPM removes the snapshots in accordance with defined retention policies.
However, only successful backups are counted towards the retention.
It is crucial to understand what a "successful backup" is.
A "successful backup" is a backup that has both completed backup and replication of all its snapshots without issues.
If a single snapshot in a policy that backups 100 volumes fails, this backup won't count.
If this continues, you may end up with a larger than expected amount of snapshots in your account and a heavier bill from AWS as a result.
By default, only backups with a status "Backup Successful" are counted towards retention:
You can change this behavior by changing the "backup is successful when" setting in the "more options" screen of a policy,:
If you'll select the second option, backups that are "Partially Successful" will now count as well:
Please keep in mind, that if you choose to ignore the output of scripts/VSS, you may end up with snapshots that are only crash consistent instead of fully consistent.
It is recommended to resolve your problems with scripts/VSS instead of ignoring them.
Here's an actual example that will explain the Generations to save better:
However, only successful backups are counted towards the retention.
It is crucial to understand what a "successful backup" is.
A "successful backup" is a backup that has both completed backup and replication of all its snapshots without issues.
If a single snapshot in a policy that backups 100 volumes fails, this backup won't count.
If this continues, you may end up with a larger than expected amount of snapshots in your account and a heavier bill from AWS as a result.
By default, only backups with a status "Backup Successful" are counted towards retention:
You can change this behavior by changing the "backup is successful when" setting in the "more options" screen of a policy,:
If you'll select the second option, backups that are "Partially Successful" will now count as well:
Please keep in mind, that if you choose to ignore the output of scripts/VSS, you may end up with snapshots that are only crash consistent instead of fully consistent.
It is recommended to resolve your problems with scripts/VSS instead of ignoring them.
Sometimes Partially Successful backups may be caused by CPM trying to backup instances that do not exist anymore: https://support.n2ws.com/portal/kb/articles/a-backup-may-be-marked-partially-successful-due-to-the-instance-probably-doesn-t-exist-anymore-error
CPM will cleanup out of retention snapshots (successful or failed) only when it has enough successful backups as you configured in your policy's retention (Generations to save).
Let's assume you have a daily policy that runs on 2 instances and is supposed to save 3 generations per instance so you expect to have 6 snapshots on EC2 after it ran 3 days and on, right?
On day 1 it runs and succeeds on both instances = 2 snapshots. Successful backup
On day 2 it runs but only the first instance is successfully backed up so now you have 3 snapshots. This is not considered as a successful backup.
On day 3 it runs again but only the first instance is successfully backed up so now you have 4 snapshots. This is not considered as a successful backup.
On day 4 it runs and succeeds = 6 snapshots (none is deleted because you only have 2 successful backups, not 3). Successful backup
On day 5 it runs and succeeds = 8 snapshots (none is deleted because you only have 3 successful backups) - on this day you'll pay for extra 2 snapshots. Successful backup
On day 1 it runs and succeeds on both instances = 2 snapshots. Successful backup
On day 2 it runs but only the first instance is successfully backed up so now you have 3 snapshots. This is not considered as a successful backup.
On day 3 it runs again but only the first instance is successfully backed up so now you have 4 snapshots. This is not considered as a successful backup.
On day 4 it runs and succeeds = 6 snapshots (none is deleted because you only have 2 successful backups, not 3). Successful backup
On day 5 it runs and succeeds = 8 snapshots (none is deleted because you only have 3 successful backups) - on this day you'll pay for extra 2 snapshots. Successful backup
Cleanup process iwll still not clean the failed backups since they're still within the retention period.
On day 6 it runs and succeeds = 6 snapshots (day 1,2 and 3 snapshots are deleted). Here, cleanup will clean the older successful ones as well as the failed ones.
On day 6 it runs and succeeds = 6 snapshots (day 1,2 and 3 snapshots are deleted). Here, cleanup will clean the older successful ones as well as the failed ones.
The good backups don't necessarily have to be one after the other but as long as you have failed backups between successful backups, the failed ones will not get deleted.
Related Articles
How cleanup works and why it might not delete snapshots older than the defined retention in CPM 3.0 and above
Background This KB Article will explain how the cleanup process works and why sometimes cleanup might not delete old snapshots It includes following sections: About the cleanup process Why Cleanup might not delete old snapshots Examples About the ...Cleanup may not delete out of retention backups (v2.1.3 only)
If cleanup does not delete out of retention backups, please search the Cleanup log for the following print: Warning - No backups were marked for deletion in policy , because out of X completed backups, only 0 were marked as successful If you found ...Snapshots may not get deleted if cleanup is running for more than 1 hour
If snapshots which are out of retention are not deleted by cleanup, please download CPM logs from GUI. Please check for the “Last Cleanup” time in “General Settings”, next to the “Run Now” button. Please search the logs for these 3 errors ...Complete_VSS operation may fail if snapshot takes more than 45 minutes to complete
Complete_VSS operation may fail if snapshot takes more than 45 minutes to complete, due to exceeding the default VSS timeout: In order to verify this, you should check the Agent's logs - VSS script should fail with return code "3" immediately after ...Daily Summary/Cleanup Log may alert of false Cleanup issues
You may receive the following warning in Daily Summary or Cleanup Log: Warning - Cleanup did not mark any backups for deletion for policy DailyPolicy, because out of 3 completed backups, only 1 were marked as successful This warning may be triggered ...