Known Issues
N2WS-17735 - EFS cleanup fails with AccessDeniedException
Issue: When trying to delete EFS backups via CPM for an AWS backup vault that was already manually deleted in AWS, it may fails with AccessDeniedException in UI and following in the agent logs: Error,Failed to delete EFS backup (recovery point ARN: ...
N2WS-13893 - Retry of backup will not delete previous backup record from monitor
Issue: The default behavior is that when a retry happens for a backup record it deletes the previous backup records and keeps only the latest try. While for some this helps with showing only the latest attempts, for other customers this can be a ...
N2WS-16181 - S3 backups may get deleted unexpectedly without being archived to Glacier
Issue summary: S3 backup policies with archive to Glacier enabled, and the S3 backups frequency equal to the frequency of moving expired S3 backups to Glacier (e.g. both set to "1"), may result in deletion of an expired S3 backup instead of archiving ...
N2WS-15684 - Cleanup issues using copy to S3
Issue summary: Policies with S3 backup enabled and backing up (1) Independent volumes, and/or (2) instances purchased from AWS marketplace May result in premature deletion of S3 backups Affected versions: v3.1.0a, v3.1.0b Patch: Patch for 3.1.0b ...
N2WS 3.1.x - cancel button not working properly
Issue summary In several components cancel operations in the UI may not take place as expected, See details below. Issue description Component/Feature Problem description Actual results Recovery window 1. Go to recovery page. 2. Click recover 3. ...
N2WS 3.x - Backup may fail with "could not get object from aws. Reason: SSL validation for <Host> [SSL: CERTIFICATE_VERIFY_FAILED]"
Issue: Running backups may fail with the following error: ERROR: run_snapshots(agent.py:1796) (instance: i-1234567) could not get object from aws. Reason: SSL validation failed for https://ec2.us-east-1.amazonaws.com/ [SSL: CERTIFICATE_VERIFY_FAILED] ...
N2WS-14705 - S3 Sync operation may not be triggered by a policy scheduler
Issue summary: S3 Sync target included in a policy will not be triggered by policy scheduler, only run ASAP will trigger an S3 Sync operation. Affected versions 3.1.0 Patch: To fix this issue, deploy the fix attached to this KB Article.
CPM v3.0.x may launch unnecessarily large Cleanup workers, resulting in extra costs
CPM v3.0.x may launch unnecessarily large Cleanup workers, resulting in extra costs. In order to reduce the costs, please use the following procedure: Connect (via ssh) to the machine running cpm (you will need to execute "sudo -i" to gain the root ...
Error "create multi-volume snapshots of instance i-123456789 failed" may occur during backup
Error: You might see this error in the cpm server logs or the UI Alerts when running backups in CPM 3.0 and above: create multi-volume snapshots of instance i-123456789 failed. Error: You are not authorized to perform this operation. Encoded ...
N2WS-12858 - CPM agent status 'never heard from' due to windows environment variable
Issue summary: If you are able to connect to the CPM UI over browser from within the windows backup target, but the Agent is still showing status 'never heard from' in the CPM UI, this can be due to an environment variable in your target Windows ...
N2WS-13787 - Duplicate row in recovery search and empty dropdown fields (e.g., VPC)
Issue summary When performaning search for an instance in the Recover window, row will show as duplicate. This can also lead to some of the values not being populated in the instance recovery popup window Issue description When performaning search ...
S3 backups may be stored for X days instead of X months
Issue description: When creating or editing a policy with S3 backup enabled and the default GUI time retention unit in the “Keep backup in S3 for at least” section is not changed, S3 backup retention time will be stored internally as “Days” instead ...
N2WS-13434 - Backup archived to Glacier disappears from the Backup Monitor
Issue summary Archive backups records might be deleted before their retention period. Issue description A backup record that contains Only archived backup(s), and Does not contain AWS snapshots nor S3 backups or cleanup process deleted all the ...
RDS Cross Region DR fails with Exception: Cannot copy more than 5 snapshots across regions
You may see the following errors in the CPM logs when cross region RDS DR copies fail to successfully copy to the DR region. ERROR: cpm_copy_dbsnapshot(aws_rds_utils.py:217) Exception: Cannot copy more than 5 snapshots across regions ...
"Cost Explorer Query Failure" alert in CPM 3.0
After upgrading to CPM 3.0, you may see an Alert in the console which says "Cost Explorer Query Failure". If you view the Alert, you will see an AccessDeniedException: The cause of this alert is due to the CPM role permissions not being updated with ...
VPC Clone may fail with "Template validation has failed"
VPC Clone may fail with the error "Template Validation Failed. Failure reason: S3 error. The bucket you are attempting to access must be addressed using the specified endpoint. Please Send all future requests to this endpoint. For more ...
File Level Recovery may fail if Elastic Load Balancing (ELB) is in use
Using FLR to explore volumes may fail if Elastic Load Balancing (ELB) is in use When using FLR to explore files you may get one of the following errors when the worker launches. This could be caused by an ELB (Elastic Load Balancing) in your ...
billingProducts code is null and YUM updates fail after restore of some SUSE and Red Hat Instances
On some operating systems, primarily SUSE and Red Hat, you will find that AWS cannot properly restore the "billingProducts" ID in the metadata for the instance and the field will be set to "null". Also, YUM repository updates will begin to fail. If ...
Cross-region and cross-account DR of an encrypted RDS cluster may fail in v2.6
Encrypted cluster RDS backups (Cross-region & cross-account) DR might fail. You may see the following in the logs Error - RDS DR copy snapshot failed for region US East (Ohio), snapshot Snapshotname (to DR account). Reason: Encrypted source snapshot: ...
S3 worker in v2.6 may fail to establish connection with the "IndexError: list index out of range message" error
Workers are failing to connect with following error: loud-init v. 18.3-9-g2e62cb8a-0ubuntu1~18.04.2 running 'modules:config' . Up 33.38 seconds. creating log dir mkfs: no device specified Try 'mkfs --help' for more information. mount: /tmp/vmb: can't ...
RDS DR may fail in v2.6.x with the "cannot be copied because it does not exist in your account" error
RDS encrypted backups failing DR according to backup logs with "cannot be copied because it does not exist in your account" error You may see the following in the logs Info - RDS DR, found matching KMS alias alias/BMS-Standard-Key (source US East (N. ...
Recovering instance in v2.6.0a (only) via REST API may result in error 500 and the "AttributeError: 'unicode' object has no attribute 'aws_region'" exception
The problem is on v2.6.0a, in REST, recovery might fail with error 500 and in the CPM UI it looks in progress. The following errors may be logged: ERROR MainThread cpmrequest.py 151 call API Request ...
Scheduled reports may not be sent with the "ClientError: An error occurred (ExpiredToken) when calling the ListVerifiedEmailAddresses operation: The security token included in the request is expired" error
Scheduled reports may not be sent with the following errors: INFO: get_ses_client(ses_management.py:72) _ses_client=<botocore.client.SES object at 0x7f41570c8e10> ERROR: get_verified_addresses(ses_management.py:92) Failed SES: ...
RDS backups may fail with "CPM Server identified this backup as orphaned: marking as failed"
The problem may happen if 2 policies are using the same schedule/target RDS, And therefore are trying to backup the same RDS databases at the same time. The following errors may be logged in the Backup Log: Info - Backup is agentless, managed by CPM ...
VPC Clone may fail with error: "Unable to get the object "
You may be presented with the following error when attempting a cross region clone of a larger sized VPC. Failed cloning one or more VPCs: [vpc-1ab34b56:S3 error: Unable to get the object ...
Error - RDS DR copy snapshot failed (...) The snapshot requires a target option group with the following options: [Timezone]
While attempting an RDS DR copy the copy may fail with an error "The snapshot requires a target option group with the following options: [Timezone]" This is due to a bug. A patch to fix this issue is available for CPM v2.5.0. NOTE This patch can ONLY ...
Cannot utilize IDP login when domain name has dash/hyphen character
When trying to login into the CPM Manager via an Identity Provider, a Domain name with a dash or hyphen in it will result in the following error: “Login Failed Username can only contain alphanumeric characters and underscores (invalid user name ...
ClientError: An error occurred (UnauthorizedOperation) when calling the DescribeVpcAttribute operation: You are not authorized to perform this operation. "cloudformation:DescribeStacks",
ClientError: An error occurred (UnauthorizedOperation) when calling the DescribeVpcAttribute operation: You are not authorized to perform this operation. "cloudformation:DescribeStacks", If you face this issue, please upgrade to latest 2.5 version ...
Patch for slow loading of backup monitor after upgrade to 2.4.0
Customers with a large number of backup history records may notice slower loading of the "Backup Monitor" view after upgrading to 2.4.0 If you are seeing this issue, please update to the latest version of CPM, v2.5.0 Reference defect number – ...
Patch for cost reduction of copy to S3
Installation of the patch will reduce costs drastically for customers who are using S3 (Between 5 to 20 times less expensive than previous version) The improvements are: For copied volumes which are larger than 512GB, worker’s volume type will be sc1 ...
Permission checks fails with Error: “User not Found”
Permission checks fails with Error: “User not Found” You may receive the following error when checking permissions on an account within CPM. Also, the policy check .CSV file will show only “User not Found” when opened. In the cpm_server logs you ...
Error when running a Permissions check post upgrade CPM 2.4 , [Errno 13] Permission denied: '/opt/n2wsoftware/scripts/permissions.json'
After upgrading to CPM 2.4 users may experience an error when on the Accounts page and clicking on 'Check AWS permissions' button may receive the following error: [Errorno 13] Permission Denied: '/opt/n2wsoftware/scripts/permissions.json' This error ...
Instance recovery may fail with the following error: "Recovery Operation Failed. Reason: EBS-optimized instances are not supported for your requested configuration."
Issue: Instance recovery may fail with an error: Recovery Operation Failed. Reason: EBS-optimized instances are not supported for your requested configuration. Please check the documentation for supported configurations. Cause: This issue may appear ...
Backup stuck in "pending" status after restart
If CPM is restarted while a backup is running, the backup will be stuck in "Pending" status in the CPM console until that policy runs again. After that, it will change to "Failed". However, the snapshots of the failed backup will still show as ...
N2WS-1040 - "Trying to download a file or directory from a backup and receive "There are no active explore sessions..." message."
When a delegate user tries to download a file or directory the following error may appear: WARNING: explore_backup_action(.\cpmserver\cpm\filemanager\views.py:551) user '%username%' is trying to download something, but there are no active explore ...
IDP/SAML integration may fail with "Failed getting CPM public host name"
Description: If you have installed the base image of CPM 2.2.0 and started using IDP/SAML authentication, and have also configured a custom hostname URL for the IDP redirect, you may get a "Error 500" in your web browser when trying to connect. If ...
CPM may fail to set the "cpm_deleted" tag to RDS Snapshots
Symptom: When performing a cross-account DR Copy of RDS snapshots, using a DR account with “Allow Deleting Snapshots” as False, the RDS snapshots that have exceeded the retention policy, will not have the “cpm_deleted” tag. You will also see the ...
The "Users" panel may become non-functional after an upgrade to v2.2.x
After upgrading to v2.2.0, customer sees empty "Users" panel: The following error appears in cpm_server.log: 2018-02-06 16:10:14,406:[139721996957440] ERROR: handle_uncaught_exception(base.py:231) Internal Server Error: /user/manage/inner/ ...
CPM may mark a successful RDS snapshot as failed
CPM may mark a successful RDS snapshot as failed. In the backup log you may see the following error: Error - snapshot of database <DBname> failed. There may also be a "rate exceeded" error in the CPM server logs: ERROR: get_list(connection.py:1180) ...
VSS Shadow Copies may not get deleted if remote Agent’s VSS script crashes
When running an application-consistent (“app-aware”), backup remote agent may not delete a VSS shadow copy if remote Agent’s VSS script crashes. The problem may be recognized by the following error messages in the remote Agent’s logs: ERROR: ...
Next page