org.apache.gobblin.util

Class HeapDumpForTaskUtils

java.lang.Object

org.apache.gobblin.util.HeapDumpForTaskUtils

public class HeapDumpForTaskUtils
extends Object

A utility class for generating script to move the heap dump .prof files to HDFS for hadoop tasks, when Java heap out of memory error is thrown.

Constructor Summary

Constructors

Constructor and Description
HeapDumpForTaskUtils()

Method Summary

Modifier and Type	Method and Description
static void	generateDumpScript(org.apache.hadoop.fs.Path dumpScript, org.apache.hadoop.fs.FileSystem fs, String heapFileName, String chmod) Generate the dumpScript, which is used when OOM error is thrown during task execution.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HeapDumpForTaskUtils

public HeapDumpForTaskUtils()

Method Detail

generateDumpScript

public static void generateDumpScript(org.apache.hadoop.fs.Path dumpScript,
                                      org.apache.hadoop.fs.FileSystem fs,
                                      String heapFileName,
                                      String chmod)
                               throws IOException

Generate the dumpScript, which is used when OOM error is thrown during task execution. The current content dumpScript puts the .prof files to the DUMP_FOLDER within the same directory of the dumpScript. User needs to add the following options to the task java.opts: -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=./heapFileName.hprof -XX:OnOutOfMemoryError=./dumpScriptFileName

Parameters:

dumpScript - The path to the dumpScript, which needs to be added to the Distributed cache. To use it, simply put the path of dumpScript to the gobblin config: job.hdfs.files.

fs - File system

heapFileName - the name of the .prof file.

chmod - chmod for the dump script. For hdfs file, e.g, "hadoop fs -chmod 755"

Throws:

IOException