A month ago, I started this topic, because my instance had been randomly shelved, despite my using it pretty regularly. It turned out the shelving was an accident, I unshelved it and it was all working fine again.
Today, I’ve gone to use my instance for the first time since being away for Christmas, and it’s shelved again! Why does this keep happening, please? Is there something I can do to protect it? It takes up quite a lot of time to have to keep waiting for it to unshelf…
The instance is on Birmingham, and the name is " Nat_nanopore_essentials_2".
First of all - apologies for this, we made a couple of mistakes that resulted in the shelving of your VM. It was unintentional, and we’re very sorry if its caused any inconvenience.
We have been trying to squeeze as much capacity for new VMs as possible from our system. Unfortunately in the case of your VM, adding one too many VMs resulted in the underlying server running out of memory and shutting down VMs to keep itself alive (with the (OOM killer)[https://www.memset.com/docs/additional-information/oom-killer/]). This meant that your VM was in a shut-down state.
Because shut-down VMs still take up an allocation of our overall capacity, we periodically go through and shelve currently shut-down VMs to reclaim (presumably) dead capacity that we can use for new VMs. Your VM was in the wrong place (shut-down) at the wrong time (we were shelving shut-down VMs), and unfortunately got shelved.
The final annoyance is that your VM was started (via Horizon, not Bryn), in a way that means it unshelves very slowly. We’ve corrected that for your VM, so it should take less than 30s or so to unshelve in the future, compared to the hour+ it took before.
Again, we’re very sorry about this series of events. We’re now being more careful about reclaiming capacity and prevent users from launching VMs from Horizon.
If you’re having any problems with your unshelved instance, please let us know.
Thanks so much for looking at this so swiftly, and for explaining what’s happened - I was wondering if my instance had somehow found its way onto some “always shelve” list!
And you’re right - the instance unshelved very quickly this time and is up-and-running (with no volume problems this time), so hopefully if this sequence of events happens again I can just quickly unshelf it and get on with it.
Thanks again, and thanks to you all for your hard work keeping CLIMB working - it’s such a useful resource to have!
Nat
