slf4 with logback causes java.lang.OutOfMemoryError: Java heap space
Hi, I swiched our project to using SLF4J / logback. All projects and build files worked (we have 40 projects) until one of ant target failed with that error: java.lang.OutOfMemoryError: Java heap space If I switch it to using slf4j-simple, I do not get that memory error. If I use log4j binding, it works too. I will start digging more, but if this raises a possible answer, I would appreciate it. This ant target runs in eclipse to invoke a junit anttask, which runs a test class that uses dbunit. -- Mel T. -- View this message in context: http://www.nabble.com/slf4-with-logback-causes-java.lang.OutOfMemoryError%3A... Sent from the Logback User mailing list archive at Nabble.com.
This issue does not ring a bell. You could use 'jmap' (new in JDK 1.6) to collect memory usage. I would be very interested in your investigation. Mel T wrote:
Hi,
I swiched our project to using SLF4J / logback. All projects and build files worked (we have 40 projects) until one of ant target failed with that error:
java.lang.OutOfMemoryError: Java heap space
If I switch it to using slf4j-simple, I do not get that memory error.
If I use log4j binding, it works too.
I will start digging more, but if this raises a possible answer, I would appreciate it.
This ant target runs in eclipse to invoke a junit anttask, which runs a test class that uses dbunit.
-- Mel T.
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch
I will use jmap and let you know. What I also found out is that the memory error goes out if I specify a logback.xml instead of relying on the default configuration. I changed it to use <configuration> <root level="warn"> </root> </configuration> and it worked fine. Ceki Gulcu wrote:
This issue does not ring a bell. You could use 'jmap' (new in JDK 1.6) to collect memory usage. I would be very interested in your investigation.
...
-- View this message in context: http://www.nabble.com/slf4-with-logback-causes-java.lang.OutOfMemoryError%3A... Sent from the Logback User mailing list archive at Nabble.com.
Mel T wrote:
I will use jmap and let you know. What I also found out is that the memory error goes out if I specify a logback.xml instead of relying on the default configuration. I changed it to use
<configuration> <root level="warn"> </root> </configuration>
and it worked fine.
...
When running ANT tasks in eclipse, it seems that the log messages are accumulated in memory before they get dumped in the console. Therefore, when running with the default config, it causes all debug logs to be in memory (and these failing tests use hibernate, spring, and many other libs). It is unclear to me though why the problem occurs with logback and not with sl4j-simple nor log4j. -- View this message in context: http://www.nabble.com/slf4-with-logback-causes-java.lang.OutOfMemoryError%3A... Sent from the Logback User mailing list archive at Nabble.com.
Mel T wrote:
When running ANT tasks in eclipse, it seems that the log messages are accumulated in memory before they get dumped in the console. Therefore, when running with the default config, it causes all debug logs to be in memory (and these failing tests use hibernate, spring, and many other libs).
It is unclear to me though why the problem occurs with logback and not with sl4j-simple nor log4j.
Indeed, that's a very interesting question. -- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch
This memory problem is caused by the JUnit and task. This implementation (1.7.0, don't know about more recent version) creates a ByteArrayOutputStream where all logs coming from the tests are sent to. All logs are accumulated in there and this buffer is freed after the test finishes. Some tests use DbUnit, which has an astronomic number of debug logs. I ran YourKit profiler on that and this ByteArrayOutputStream grew to 1G at one point around which it caused the heap space memory error. When running with no logback.xml, the default behavior kicked in and all debugs ended up in memory. Using slf4 simple binding, only INFO and up is displayed. So the DBUnit debug logs are not accumulating. Using log4j binding without config file also seems to only show INFO and up. -- Mel T. Mel T wrote:
When running ANT tasks in eclipse, it seems that the log messages are accumulated in memory before they get dumped in the console. Therefore, when running with the default config, it causes all debug logs to be in memory (and these failing tests use hibernate, spring, and many other libs).
It is unclear to me though why the problem occurs with logback and not with sl4j-simple nor log4j.
-- View this message in context: http://www.nabble.com/slf4-with-logback-causes-java.lang.OutOfMemoryError%3A... Sent from the Logback User mailing list archive at Nabble.com.
Mel T wrote:
This memory problem is caused by the JUnit and task.
I suppose you meant "by the Junit *Ant* task".
This implementation (1.7.0, don't know about more recent version) creates a ByteArrayOutputStream where all logs coming from the tests are sent to. All logs are accumulated in there and this buffer is freed after the test finishes.
Some tests use DbUnit, which has an astronomic number of debug logs. I ran YourKit profiler on that and this ByteArrayOutputStream grew to 1G at one point around which it caused the heap space memory error.
Ideally, you would contact the Ant user list to report this problem. The DBUnit folks should be notified as well. (Generating 1GB of logs may not be reasonable.)
When running with no logback.xml, the default behavior kicked in and all debugs ended up in memory.
Using slf4 simple binding, only INFO and up is displayed. So the DBUnit debug logs are not accumulating.
Makes sense.
Using log4j binding without config file also seems to only show INFO and up.
Since log4j does not have default (valid) configuration, it just complains that there is no config file and proceeds to drop all log messages. I am sorry about the time you spent/wasted on this issue. Thank you for sharing the results of your investigation on this list.
-- Mel T.
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch
Some tests use DbUnit, which has an astronomic number of debug logs. I ran YourKit profiler on that and this ByteArrayOutputStream grew to 1G at one point around which it caused the heap space memory error.
One thing I also noticed when looking at the profiler was that the number of LoggingEvent instances increased dramatically as well before they were eventually garbage collected. Maybe that is just a normal thing though.
Ideally, you would contact the Ant user list to report this problem. The DBUnit folks should be notified as well. (Generating 1GB of logs may not be reasonable.)
I will do that.
I am sorry about the time you spent/wasted on this issue. Thank you for sharing the results of your investigation on this list.
I did not spend that much time. Did that in between a few things. The worst is getting to remember how to use and interpret YourKit. This is not something I do every day so I forget. -- View this message in context: http://www.nabble.com/slf4-with-logback-causes-java.lang.OutOfMemoryError%3A... Sent from the Logback User mailing list archive at Nabble.com.
Mel T skrev:
Hi,
I swiched our project to using SLF4J / logback. All projects and build files worked (we have 40 projects) until one of ant target failed with that error:
java.lang.OutOfMemoryError: Java heap space
If I switch it to using slf4j-simple, I do not get that memory error.
If I use log4j binding, it works too.
Sounds like you are close to the memory limit, and that logback is just that bit larger. It is perfectly possible I guess wrong. Can you run the ant task in a single JVM which you invoke with a memory profiler? -- Thorbjørn Ravn Andersen "...plus... Tubular Bells!"
participants (3)
-
Ceki Gulcu -
Mel T -
Thorbjoern Ravn Andersen