CPM may fail to communicate with AWS gateways with the "timed out" error

CPM may fail to communicate with AWS gateways with the "timed out" error during various operations:

ERROR: run_snapshots(.\cpmagent\agent.py:1462) snapshot of policy PolicyName, volume: vol-0123456789abcdef failed (independent). Reason: timed out

ERROR: wait_for_snapshots(.\cpmagent\agent.py:514) ec2, Policy 4, error in 'get_all_snapshots()': timed out

...

ERROR: run_snapshots(.\cpmagent\agent.py:1179) (instance: i-0123456789abcdef) could not get object from aws. Reason: timed out Exception: timed out

...

ERROR: handle_boto3_error_exception(.\common\aws_utils.py:514) Unknown Exception: 'NoneType' object has no attribute '__getitem__' (original: HTTPSConnectionPool(host='ec2.us-west-1.amazonaws.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<botocore.awsrequest.AWSHTTPSConnection object at 0x7fc0e3d37350>, 'Connection to ec2.us-west-1.amazonaws.com timed out. (connect timeout=60)')))

...

ERROR: delete_all_snapshots_of_a_backup_record(.\cpmserver\cpm\aws_utils.py:432) Failed to delete snapshot snap-abcdef0123456789 (policy PolicyName), delete operation failed. Reason: timed out (error: None)

This is an indication that CPM instance can't reliably communicate with AWS gateways, most likely due to a slow/misconfigured proxy/firewall/NAT gateway etc. For troubleshooting purposes, you may switch to a direct connection - CPM will be able to operate normally.

Related Articles
CPM may malfunction if security group is misconfigured
CPM may malfunction if security group is misconfigured. For example CPM may experience slow response and timeout in CPM console/GUI when adding backup targets: ERROR: policy_add_databases(.\cpmserver\cpm\views.py:3415) rds.RDSConnection or ...
CLI operations may fail with the “The read operation timed out” error
If you receive the “The read operation timed out” error during a CLI operation, you can use the parameter "--request_timeout<value>" to set a longer timeout. For example, "--request_timeout=60" would set a 10 minute timeout. --request_timeout ...
N2WS 3.x - Backup may fail with "could not get object from aws. Reason: SSL validation for <Host> [SSL: CERTIFICATE_VERIFY_FAILED]"
Issue: Running backups may fail with the following error: ERROR: run_snapshots(agent.py:1796) (instance: i-1234567) could not get object from aws. Reason: SSL validation failed for https://ec2.us-east-1.amazonaws.com/ [SSL: CERTIFICATE_VERIFY_FAILED] ...
Old versions (pre-v2.1.x) of CPM may fail to communicate with AWS with error "could not get object from aws" and reason "SSL3_GET_SERVER_CERTIFICATE:certificate verify failed"
Old versions (pre-v2.1.x) of CPM may fail to communicate with AWS, logging the following error: Error - (instance: i-12345678) could not get object from aws. Reason: [Errno 1] _ssl.c:503: error:14090086:SSL ...
Internet access required for CPM operations
CPM Server: All outbound CPM communication is to the 443 port (https) For ongoing operations for a production CPM server it needs to access AWS endpoints to use AWS APIs. You can decide to expose only relevant regions. AWS endpoint information can be ...

CPM may fail to communicate with AWS gateways with the "timed out" error

CPM may fail to communicate with AWS gateways with the "timed out" error

Related Articles

CPM may malfunction if security group is misconfigured

CLI operations may fail with the “The read operation timed out” error

N2WS 3.x - Backup may fail with "could not get object from aws. Reason: SSL validation for <Host> [SSL: CERTIFICATE_VERIFY_FAILED]"

Old versions (pre-v2.1.x) of CPM may fail to communicate with AWS with error "could not get object from aws" and reason "SSL3_GET_SERVER_CERTIFICATE:certificate verify failed"

Internet access required for CPM operations