(If the S3 copy fails without producing the above error message, run the copy task again, then enable termination protection)
If the root volume has been retained from the failure, first find the detached worker volume in EBS. The volume ID can be found in the backup log.
If the volume cannot be found, double check the session in Backup Manager shows "root volume has been retained" and the volume ID is not mistyped in AWS.
Next, attach this volume to the CPM server instance or other Linux instance.
the CPM server instance cannot be found when attaching the volume,
check the region and availability zone of the worker volume match the CPM
Once attached, go to step 3.
Step 2b: Create a copy of the termination protected worker's volume and attach :
In most cases this should only be done after the S3 copy shows as failed in the CPM console.
1. Find the worker appliance in AWS and click its Root-device from the description, then click the EBS ID.
2. Create a snapshot of the volume and go to the snapshot by following the link on the created snapshot dialog:
3. Create a volume from this snapshot. While creating the volume be aware of the availability zone, it will need to match the CPM server or Linux server being used to mount the volume.
Once the volume has finished creating, click the link on the "Request Succeeded" page.
4. Finally, attach the volume to the CPM server or other Linux instance of your choosing and go to Step 3.
Step 3: Mount the volume within Linux:
the worker volume to the CPM server will allow retrieving the logs using "download logs"
in the CPM UI, please follow the below steps carefully
Once attached, the volume needs to be mounted within the instance.
The below bash commands are specific to the CPM server
- Using an SSH client of your choosing, make a connection and log into the CPM server.
The CPM server uses key authentication (key selected in AWS when the CPM server was first launched) and user: cpmuser
- Use the following command to find the volume:
- The S3 worker partition will be "part" in the TYPE column and will have a blank MOUNTPOINT,
The correct partition is "xvdg1" in this example.
Note: the partition name may be different on your CPM server.
Be sure to check you have found the correct partition!
- Create a mount point for the worker partition by running these commands:
sudo mkdir /worker
- Now mount the worker volume to the CPM server with the following command, replace "<partition name from 3.>" with the name of the partition found in 3. above:
sudo mount /dev/<partition name from 3.> /worker
Confirm the mount by going to /worker
and listing the directory. You should see the file system of the worker
Step 4: Getting worker logs from the mounted volume:
- Compress the entire log folder
sudo tar -zcvf /tmp/workerlogpack.tar.gz /worker/var/log
- Find workerlogpack.tar.gz in /tmp
At this point, workerlogpack.tar.gz can be copied from the instance using FTP or another method, and this log package can be attached to the support case. See below for clean up.
Alternatively, if the worker volume has been mounted to the CPM server, its possible to copy this log pack to the CPM log directory, then retrieve the log package in the UI.
- To copy the log package from /tmp to the CPM log directory run this command:
sudo cp /tmp/workerlogpack.tar.gz /var/log/cpm/workerlogpack.tar.gz
The copy can be confirmed with this command:
- Next, generate the log package from the CPM server by clicking "download logs"
- Finally, upload this CPM log package to the support case.
The next steps show how to remove the mount and clean up created files.
Step 5: Clean up:
- Unmount the volume with these commands:
sudo umount /dev/xvdg1
- Remove the mount point folder with this command:
sudo rmdir /worker
- Remove the temporary log package with this command:
sudo rm /tmp/workerlogpack.tar.gz
- If logs were copied to the CPM server log directory, remove the log package with this command:
sudo rm /var/log/cpm/workerlogpack.tar.gz
- The worker root volume can now be detached from the instance in AWS. You can delete this volume, and any snapshots created from the worker appliance.
- If termination protection was enabled, disable it and terminate the CPM worker appliance.