I have a customer who called me yesterday for having a task in vCloud Director that has been stuck in the cancelling stage for over a month. The problem has started after a power outage they had in their datacenter. vCloud Director is operating just fine, just that particular task was doing nothing and in reality it was cancelled but in the vCloud GUI it kept saying “Cancelling Copy” as shown in the below screenshot.
My customer was bugged by this and wanted to get rid of this message in his vCloud Director GUI, which we were able to do successfully by following the below procedure. Please note the following procedure will take your vCloud Director Portal down for few minutes, so you probably want to schedule it during an outage. At a high level what is the below procedure will be doing is gracefully shutting down the vCD Cells then bringing them up, which should clean up any stuck cancellation tasks. Below is a step by step of how to do so:
- Log in to the target vCloud Director cells using SSH as root.
- Navigate to $VCLOUD_HOME/bin/ (typically /opt/vmware/cloud-director /bin/ ).
- Retrieve the current job status. This cell-management-tool command line supplies system administrator credentials and returns the count of running jobs (i.e. Job count 3, Is Active = true):
./cell-management-tool -u administrator -p Pa55w0rd cell –status
- Stop the task scheduler to quiesce the cell.Use this cell-management-tool command:
When the Job count is 0 and Is Active is false, it is safe to shut down the cell. Keep executing the command in step 4 till these conditions are met then use the below command to stop the Cell:
service vmware-vcd stop
Note: You can run “service vmware-vcd status” to ensure that the vcd cell was stopped properly.
- Repeat the above steps to other vCloud Directors Cells involved in your setup
- Start your vCD Cells again using the following command (Repeat this for all of your vCD Cells): service vmware-vcd start
Now you should be able to login back to your vCloud Director Cells and should have all these Stucked While Canceling tasks vanished from your web interface. Hope you find this useful!