Pilon Java Heapspace?


#1

Please use this template to help us fix your problem quickly! Please don’t include sensitive information (including IP addresses) and fill in sections as appropriate.

CLIMB SERVICE PROBLEMS (Bryn, OpenStack, Launching VMs, Creating and attaching Volumes, etc.)

1. Describe the problem. Are there any error messages you can paste? Can you provide a screenshot of the problem (with sensitive information removed)?

I have a student using Galaxy to see if Pilon can be used to resolve the ends of Streptomyces chromosomes.
She has used Spades to assemble and BWA to map the reads back onto the assembly.
We run into what seems like a Java Heapspace issue - is it possible to allocate more memory?
Streptomyces are among the largest bacterial genomes, so i’m not entirely surprised - assuming i’m right?!?
Pilon gives the following error which sounds like a Java memory issue:

Fatal error: Exit code 1 ()
Exception in thread “main” java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simontuffs.onejar.Boot.run(Boot.java:340)
at com.simontuffs.onejar.Boot.main(Boot.java:166)
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.broadinstitute.pilon.GenomeRegion.processBam(GenomeRegion.scala:275)
at org.broadinstitute.pilon.GenomeFile$$anonfun$processRegions$4$$anonfun$apply$3.apply(GenomeFile.scala:112)
at org.broadinstitute.pilon.GenomeFile$$anonfun$processRegions$4$$anonfun$apply$3.apply(GenomeFile.scala:112)
at scala.collection.immutable.List.foreach(List.scala:381)
at org.broadinstitute.pilon.GenomeFile$$anonfun$processRegions$4.apply(GenomeFile.scala:112)
at org.broadinstitute.pilon.GenomeFile$$anonfun$processRegions$4.apply(GenomeFile.scala:109)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at scala.collection.parallel.ParIterableLike$Foreach.leaf(ParIterableLike.scala:972)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.scala:49)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51)
at scala.collection.parallel.ParIterableLike$Foreach.tryLeaf(ParIterableLike.scala:969)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.internal(Tasks.scala:169)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:443)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.compute(Tasks.scala:149)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:443)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:160)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

2. What is the name of the VM or volume that the problem is affecting?

iStrep

3. Where is the VM or volume hosted; Birmingham, Cardiff, Swansea or Warwick?

Warwick

4. Is there any other information that you think might be helpful?

SOFTWARE OR APPLICATION PROBLEMS (Python can’t find the modules I installed, Prokka doesn’t predict any ORFs, etc.)

1. Describe the problem. Include links to any guides that you might be following and a brief history of the commands that you’ve run. Paste any error messages in code-blocks (paste, highlight and press Ctrl+Shift+C).

2. What operating system are you running; GVL, Ubuntu 16.04, Centos 7, etc.?
MacOS

3. Is there any other information that you think might be helpful?

Pilon requirements:
Pilon is distributed as a single jar file. Requirements to run it include:

  • Java runtime 1.7 or later
  • 8GB or more memory to allocate to the JVM. The amount of memory required depends on the genome, the read data, and how many fixes Pilon needs to make. Generally, bacterial genomes with ~200x of Illumina coverage will require at least 8GB, though 16GB is recommended.
  • Larger genomes will require more memory to process; exactly how much is very data-dependent, but as a rule of thumb, try to allocate 1GB per megabase of input genome to be processed.

#2

You could try the solution presented in this thread - https://github.com/broadinstitute/pilon/issues/44

If you still need a larger amount of RAM then you could try the analysis on a group instance (assuming you are using a user instance).

All the best,
Sion


#3

Thanks for that.
Can you imagine a way that that might work for a Climb Galaxy instance?

I noticed that there is a Galalxy login option via Lubuntu…


#4

If using a larger instance does not work I would recommend contacting the Galaxy or Pilon dev team as they will be much more knowledgeable about this issue.


#5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.