Unshelved VM I was using was stopped and is now not accessible


#1

I requested that my vm (137.44.59.161) not be shelved as I am running anayses on it that has been going for over a week now. I checked it yesterday and it was still running, but have been unable to ssh in today. I checked it on the advanced interface and it looks like this instance had been stopped this morning (shelved??) - the requestID was req-d10cb0c2-3fa9-4fe4-9b76-fccfcae9c0ae and was listed at 5am. I hace since tried to start it again but still cannot ssh in. There is a volume of data attached to this vm and the analyses are precious - any way this can be fixed?? The vm is hosted at Swansea…


#2

Hello,

Unfortunately you’ve been the victim of some hardware failure at Swansea, at approx 04:51:25 the hardware your VM was on died.

I see you started your VM back up and you should be able to SSH in now.

There’s not a whole lot that can be done as hardware failure is largely unpredictable.

Thanks,
Imran


#3

That is unfortunate. It has set my VM back to a version from a few months ago :frowning:

I guess there is no way of preventing this from happening in future?


#4

Not from the hardware failure point of view.

Potentially you could alter your workflow so there are more ‘checkpoints’ so to speak, so instead of one big job, split it up into smaller job, then if there’s an issue, you might lose a smaller bit.


#5

If when you say your VM has been set back to a version a few months ago, you might want to check any external volumes you had mounted and if they are still mounted.


#6

The volume I had mounted is still mounted and accessible, but the data on it is from a few weeks ago (not months, I now realised).

I will definitely start backing this intermediate data up ontoa hard drive at more regular intervals, but unfortunately the job that was running on multiple datasets had been running for 10 days and hadn’t finished.

Thanks for keeping me in the loop


#7

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.