You may encounter the message "Worker did not establish connection" in the log of an S3 copy or S3 restore operation.
Day mm/dd/yyyy HH:MM:SS - Error - Worker i-1234567890abcdef did not establish connection - terminating operation
Or you may see, the s3 copy task with the status "In progress (0%)" for an extended period of time (6+ hours), until the task fails by timeout.
The first message indicates the S3 copy worker instance has failed to establish a network connection to the CPM server,
Stuck in progress usually indicates the S3 copy worker could not connect to the S3 bucket.
Check the following:
1. The worker appliance security group settings, the following must be allowed for both backup and restore:
INBOUND HTTPS (port 443):
- To the CPM server private or public IP.
OUTBOUND HTTPS (port 443):
Note 2: If restoring from S3 to a different region, select a security group at the target region which allows outbound connections to the source region S3 bucket. Please see 4. below as well.
2. The CPM server security group settings:
INBOUND and OUTBOUND HTTPS (port 443):
- To the subnet the worker is configured to use.
Also be sure access to the CPM server is allowed via port 443 where necessary.
4. The worker will connect to S3 via internet, even if the CPM server and S3 bucket are in the same region.
- The worker must have a public IP assigned by the subnet; -or- the subnet must be configured with a NAT gateway, for the worker to communicate bi-directionally with S3.
- Only if the EBS backups and S3 bucket are in the same region, this can be worked around by configuring an S3 endpoint as shown here: https://n2ws.com/support/documentation/appendix-s3-configuration
5. The worker must be able to resolve the DNS name of the region with your S3 bucket. (e.g. https://s3.us-west-2.amazonaws.com/
). Please ensure the worker will be able to resolve DNS names of S3 endpoints.
6. If an HTTP proxy is necessary to access public IPs, ensure this is
properly configured in the worker settings and not blocking the transfer
to s3. It may be advisable to test by opening internet to the worker
without a proxy in some cases to be sure the proxy is not causing an