-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 OK so I am working on a custom job engine and I am trying to separate my job logs from the main application logs as well as separating the logs for each individual job run from each other. My plan is to use the sifting log appender to put the logs into a hierarchy on the filesystem as the {job_group}/{job_name}/{instance_id} I know I can pull multiple values out of the MDC for use in the pattern layout but I am having trouble using them in the filename. I am not sure my above approach will work for directory names but according to the docs it should work at least for the filename. My second question is how do I keep my job logs out of the main log? Here is my current config: <?xml version="1.0" encoding="UTF-8"?> <configuration> <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender"> <!-- encoders are assigned the type ch.qos.logback.classic.encoder.PatternLayoutEncoder by default --> <encoder> <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%thread] %C:%L %msg%n</pattern> </encoder> </appender> <appender name="jobLoggerAppender" class="ch.qos.logback.classic.sift.SiftingAppender"> <sift> <appender name="FILE-${jobId}" class="ch.qos.logback.core.FileAppender"> <file>${jobGroup}-${jobId}.log</file> <layout class="ch.qos.logback.classic.PatternLayout"> <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%thread] %C:%L %msg%n</pattern> </layout> </appender> </sift> </appender> <logger name="com.lordofthejars.foo" level="INFO" additivity="false"> <appender-ref ref="STDOUT" /> </logger> <!-- Strictly speaking, the level attribute is not necessary since --> <!-- the level of the root level is set to DEBUG by default. --> <root level="DEBUG"> <appender-ref ref="STDOUT" /> </root> <logger name="jobLogger" level="debug"> <appender-ref ref="jobLoggerAppender"/> </logger> </configuration> - -- Cheers, Matt Young solid@youngdev.net http://youngdev.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSV1HSAAoJEO5jycTTPEzc/0IP/Rr48HksJFgSDg2iSUIsOL6X arJ/VADWDWG+4ydApy+Gw5B5Zx1D8Ds6TQzKB9rSX2lXOJoO/zRb/ObF6eOx8d9t koVSO7bG9foID/DUm29cGm7ne11yA06EmYyxjJQ0e8TD6ixxIPyiQIOr1cfvRT0/ e3mkrH+xJUByXmRu9elsOzDYk1xqwIRY6I5KcB1g3/2NxOTjG8DL7n3WTcKbea0t g0QOlFv5jLwsxa5l0SDn2ONtV+QipS/ecvTdnH+bk7FvY5bX0UcEURfQcKiMK9Sl fozS+AeukrBi4W72iVFYeQxxaYpJyEqRp457yq++wPGbeJP8w0cn7Cke9a8i9M7h pudRazu6MXF1u9c2kWscCugdpHAcEA+HXS94Ah5RlXkX9VBqKoc2pjrzx+wrZcCh nAODUWdFx8C654hmjFSD0oUKNaG5WEwJbQaAwFCrOdlB31Teeqw10e3ArcI/D9OL dnWFXGdD/QKKoCfgMCs3nFhPDmGuMzcBDopR5bxF5DDspoALxiHbLS6eRIXhYH+i aieeERi4GnJsCRfCVJbqK4qOq2ePuwi8HDptRNTv8HiHmC4uNz51k/VL0cqHpjlw 8ZrSUFAe4h6D7TsrLJ9ehac8J1QlAtycq1MX/4fleWXf2dK53kgNBbPDNBQsV2lZ X34Ooo+zOV15p7ijQ2en =mdwF -----END PGP SIGNATURE-----
Matt, You can do this. I've done it. First thing to note is that sifting appender only works off one key. So in your code concatenate the folder, sub folder and filename into one value and stick that in the MDC and use it as the key to the sifter and the variable in the path. Next to make the log items only appear in one log I used two filters. One on the main log and one of the sifter. Make sure the two filters work opposed to each other. So when one accepts the other rejects. Ideally there would be a switching filter, but it doesn't exist. David
On 11 Oct 2013, at 02:18, Matt Young <solid@youngdev.net> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
OK so I am working on a custom job engine and I am trying to separate my job logs from the main application logs as well as separating the logs for each individual job run from each other. My plan is to use the sifting log appender to put the logs into a hierarchy on the filesystem as the {job_group}/{job_name}/{instance_id} I know I can pull multiple values out of the MDC for use in the pattern layout but I am having trouble using them in the filename. I am not sure my above approach will work for directory names but according to the docs it should work at least for the filename.
My second question is how do I keep my job logs out of the main log?
Here is my current config: <?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender"> <!-- encoders are assigned the type ch.qos.logback.classic.encoder.PatternLayoutEncoder by default --> <encoder> <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%thread] %C:%L %msg%n</pattern> </encoder> </appender> <appender name="jobLoggerAppender" class="ch.qos.logback.classic.sift.SiftingAppender"> <sift> <appender name="FILE-${jobId}" class="ch.qos.logback.core.FileAppender"> <file>${jobGroup}-${jobId}.log</file> <layout class="ch.qos.logback.classic.PatternLayout"> <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%thread] %C:%L %msg%n</pattern> </layout> </appender> </sift> </appender>
<logger name="com.lordofthejars.foo" level="INFO" additivity="false"> <appender-ref ref="STDOUT" /> </logger>
<!-- Strictly speaking, the level attribute is not necessary since --> <!-- the level of the root level is set to DEBUG by default. --> <root level="DEBUG"> <appender-ref ref="STDOUT" />
</root> <logger name="jobLogger" level="debug"> <appender-ref ref="jobLoggerAppender"/> </logger> </configuration>
- -- Cheers, Matt Young solid@youngdev.net http://youngdev.net
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBAgAGBQJSV1HSAAoJEO5jycTTPEzc/0IP/Rr48HksJFgSDg2iSUIsOL6X arJ/VADWDWG+4ydApy+Gw5B5Zx1D8Ds6TQzKB9rSX2lXOJoO/zRb/ObF6eOx8d9t koVSO7bG9foID/DUm29cGm7ne11yA06EmYyxjJQ0e8TD6ixxIPyiQIOr1cfvRT0/ e3mkrH+xJUByXmRu9elsOzDYk1xqwIRY6I5KcB1g3/2NxOTjG8DL7n3WTcKbea0t g0QOlFv5jLwsxa5l0SDn2ONtV+QipS/ecvTdnH+bk7FvY5bX0UcEURfQcKiMK9Sl fozS+AeukrBi4W72iVFYeQxxaYpJyEqRp457yq++wPGbeJP8w0cn7Cke9a8i9M7h pudRazu6MXF1u9c2kWscCugdpHAcEA+HXS94Ah5RlXkX9VBqKoc2pjrzx+wrZcCh nAODUWdFx8C654hmjFSD0oUKNaG5WEwJbQaAwFCrOdlB31Teeqw10e3ArcI/D9OL dnWFXGdD/QKKoCfgMCs3nFhPDmGuMzcBDopR5bxF5DDspoALxiHbLS6eRIXhYH+i aieeERi4GnJsCRfCVJbqK4qOq2ePuwi8HDptRNTv8HiHmC4uNz51k/VL0cqHpjlw 8ZrSUFAe4h6D7TsrLJ9ehac8J1QlAtycq1MX/4fleWXf2dK53kgNBbPDNBQsV2lZ X34Ooo+zOV15p7ijQ2en =mdwF -----END PGP SIGNATURE----- _______________________________________________ Logback-user mailing list Logback-user@qos.ch http://mailman.qos.ch/mailman/listinfo/logback-user
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I hear ya on the sifting appender only accepting 1 key. And I changed the jobId to MDC.put("fname",jobGroup+"/"+jobName+"/"+jobId ); which does work with the following configuration: <appender name="jobLoggerAppender" class="ch.qos.logback.classic.sift.SiftingAppender"> <discriminator class="ch.qos.logback.classic.sift.MDCBasedDiscriminator"> <key>fname</key> <defaultValue>unknown</defaultValue> </discriminator> <sift> <appender name="SIFT_INTERNAL" class="ch.qos.logback.core.FileAppender"> <prudent>true</prudent> <file>${fname}.log</file> <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%thread] %C:%L %msg%n</pattern> </encoder> </appender> </sift> </appender> What concerns me is that I don't seem to be able to override the MDCAdapter that logback uses for the sifting. The documentation says that the class="..." attribute should override the MDCAdapter implementation used. but it doesn't. No matter what I put in that class attribute, it always uses ch.qos.logback.classic.util.LogbackMDCAdapter I looked at the source for that MDCAdapter implementation and have confirmed that it does only use one value but that the MDCBasicAdapter from slf4j supports multiple keys. What is preventing me from using the better implementation from SLF4J? On 10/11/2013 05:35 AM, David Roussel wrote: - -- Cheers, Matt Young solid@youngdev.net http://youngdev.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSWGNbAAoJEO5jycTTPEzccQEP/29NN/dUMkxhik0ya+w9sebW SOEZMQvfltWEEuf+tCLR/dfaVobvPDq6tMHtE5vV+p0Eve/RLaoxBemiDr/Lp14i AfAgDn5h+LFNMpGsL9QwiOJnM/uVn/yefiwTc4yeRfNTgF/aR52i6EVyRLy3b4fN jfkieohJ0BftXhUuLWpBcdDJUiIYK6w9YLrK1Hu5/LgCmmYcQPramddDsv1mrTGm /NicfKMiMriSALL2kRJY+/v2mnWmSzTWprXA30k9WM0q0cKWDQJuhqCnuU8CQ22M 3VablhdkuZE+tSMmcAcm0T6lp8fBtfvIM8d5G4vXAVcHx4c6yx96txUjgz2yAFdg 03/ge5YBVux2qWdfY+9jNbDWpKyOzb5MZ8X/VWbmAr0uSuIyoJUo76HFTH2hq8mn 5JfNmLhTNJiLp0qMFpSmaarw20ligO1PRn4N2EqAGlNUoGG43wnyMn+inGxJjVmJ cHN26KpetRHAFj5lJ/ggfP2Khr8Fn637yoEyzmtB6PQPTmj0RFZkz8DjBxIRDETU tw/VYfC+VtJyzaNG1JtfDOjErbA6VpywpzHhrklB+jaQX1nGGDfzOsndmU13R8X/ b7WziN3+ptatd7ErcAecFeOR5sxTj3nI/qpYgilZmlfU68F+XA7foeCp1VPw8DqX m4w9WsFmFPrUUfuwZAfA =HuWJ -----END PGP SIGNATURE-----
To be honest Matt I found the best way to understand these things is to set some breakpoints and step through it in the debugger. If you find the problem the your can create a pull request on GitHub. I had problems with sifting appender before and was able to get some pull requests accepted to fix my problem. David
On 11 Oct 2013, at 21:45, Matt Young <solid@youngdev.net> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I hear ya on the sifting appender only accepting 1 key. And I changed the jobId to MDC.put("fname",jobGroup+"/"+jobName+"/"+jobId ); which does work with the following configuration: <appender name="jobLoggerAppender" class="ch.qos.logback.classic.sift.SiftingAppender"> <discriminator class="ch.qos.logback.classic.sift.MDCBasedDiscriminator"> <key>fname</key> <defaultValue>unknown</defaultValue> </discriminator> <sift> <appender name="SIFT_INTERNAL" class="ch.qos.logback.core.FileAppender"> <prudent>true</prudent> <file>${fname}.log</file> <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%thread] %C:%L %msg%n</pattern> </encoder> </appender> </sift> </appender>
What concerns me is that I don't seem to be able to override the MDCAdapter that logback uses for the sifting. The documentation says that the class="..." attribute should override the MDCAdapter implementation used. but it doesn't. No matter what I put in that class attribute, it always uses ch.qos.logback.classic.util.LogbackMDCAdapter
I looked at the source for that MDCAdapter implementation and have confirmed that it does only use one value but that the MDCBasicAdapter from slf4j supports multiple keys. What is preventing me from using the better implementation from SLF4J?
On 10/11/2013 05:35 AM, David Roussel wrote:
- -- Cheers, Matt Young solid@youngdev.net http://youngdev.net
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBAgAGBQJSWGNbAAoJEO5jycTTPEzccQEP/29NN/dUMkxhik0ya+w9sebW SOEZMQvfltWEEuf+tCLR/dfaVobvPDq6tMHtE5vV+p0Eve/RLaoxBemiDr/Lp14i AfAgDn5h+LFNMpGsL9QwiOJnM/uVn/yefiwTc4yeRfNTgF/aR52i6EVyRLy3b4fN jfkieohJ0BftXhUuLWpBcdDJUiIYK6w9YLrK1Hu5/LgCmmYcQPramddDsv1mrTGm /NicfKMiMriSALL2kRJY+/v2mnWmSzTWprXA30k9WM0q0cKWDQJuhqCnuU8CQ22M 3VablhdkuZE+tSMmcAcm0T6lp8fBtfvIM8d5G4vXAVcHx4c6yx96txUjgz2yAFdg 03/ge5YBVux2qWdfY+9jNbDWpKyOzb5MZ8X/VWbmAr0uSuIyoJUo76HFTH2hq8mn 5JfNmLhTNJiLp0qMFpSmaarw20ligO1PRn4N2EqAGlNUoGG43wnyMn+inGxJjVmJ cHN26KpetRHAFj5lJ/ggfP2Khr8Fn637yoEyzmtB6PQPTmj0RFZkz8DjBxIRDETU tw/VYfC+VtJyzaNG1JtfDOjErbA6VpywpzHhrklB+jaQX1nGGDfzOsndmU13R8X/ b7WziN3+ptatd7ErcAecFeOR5sxTj3nI/qpYgilZmlfU68F+XA7foeCp1VPw8DqX m4w9WsFmFPrUUfuwZAfA =HuWJ -----END PGP SIGNATURE-----
_______________________________________________ Logback-user mailing list Logback-user@qos.ch http://mailman.qos.ch/mailman/listinfo/logback-user
participants (2)
-
David Roussel -
Matt Young