Hello,
My nanopolish jobs are failing I think due to not enough memory. I have a few questions - sorry they are pretty basic ones!
How can I check how much memory a job is using/has used in my VM?
How can I add more memory capacity to this job?
I have been increasing thread number but I’m not sure this is what I should be doing?
How many cores/threads are available with what I have - I feel like I should know where to find this info but I can’t quite remember!
Many thanks
Use the htop command to see a list of running processes sorted by their cpu utilisation, alongside an overall view of how much RAM is currently being used
Unlike queued HPC-type systems, because you are working on a VM, all of the RAM allocated to that VM is available to all jobs and the kernel is left to decide which job gets allocated what RAM. The kernel does a pretty good job of this generally, but if a job consumes more RAM than is available (32 Gb on a “user-sized” instance, and 64 Gb on a “group-sized” instance), it will kill the job with the Out Of Memory (OOM) Killer to protect itself from locking up and rebooting.
The only way to increase the RAM available to your analyses is to launch a climb.group instance if you are using a climb.user instance. Otherwise, you will need to make a request for a bigger instance with more RAM, which we can offer to projects with specific (short-term) timescales.
This is described here, and I’ve briefly mentioned it above. Additionally, there are lots of ways to check resources from inside an instance…
| Command | |
|---|---|
free -m |
Shows total/free memory in Megabytes |
free -g |
Shows total/free memory in Gigabytes |
ps auxf |
Shows consumption of cpu and RAM resources for all jobs, |
top -c |
Fancy (and informative) ps auxf. Dynamically shows total and per-job RAM use. Quit by pressing q. |
htop |
Fancy (and informative) top -c. Quit by pressing q
|
cat /proc/cpuinfo |
Shows everything the OS knows about each CPU core. |
Any combination of these should help you understand total resources and those being currently consumed.
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.