October 2013 - logback-user

Job Logger configuration
by Matt Young 12 Oct '13

12 Oct '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 OK so I am working on a custom job engine and I am trying to separate my job logs from the main application logs as well as separating the logs for each individual job run from each other. My plan is to use the sifting log appender to put the logs into a hierarchy on the filesystem as the {job_group}/{job_name}/{instance_id} I know I can pull multiple values out of the MDC for use in the pattern layout but I am having trouble using them in the filename. I am not sure my above approach will work for directory names but according to the docs it should work at least for the filename. My second question is how do I keep my job logs out of the main log? Here is my current config: <?xml version="1.0" encoding="UTF-8"?> <configuration> <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">  <encoder> <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%thread] %C:%L %msg%n</pattern> </encoder> </appender> <appender name="jobLoggerAppender" class="ch.qos.logback.classic.sift.SiftingAppender"> <sift> <appender name="FILE-${jobId}" class="ch.qos.logback.core.FileAppender"> <file>${jobGroup}-${jobId}.log</file> <layout class="ch.qos.logback.classic.PatternLayout"> <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%thread] %C:%L %msg%n</pattern> </layout> </appender> </sift> </appender> <logger name="com.lordofthejars.foo" level="INFO" additivity="false"> <appender-ref ref="STDOUT" /> </logger>   <root level="DEBUG"> <appender-ref ref="STDOUT" /> </root> <logger name="jobLogger" level="debug"> <appender-ref ref="jobLoggerAppender"/> </logger> </configuration> - -- Cheers, Matt Young solid(a)youngdev.net http://youngdev.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSV1HSAAoJEO5jycTTPEzc/0IP/Rr48HksJFgSDg2iSUIsOL6X arJ/VADWDWG+4ydApy+Gw5B5Zx1D8Ds6TQzKB9rSX2lXOJoO/zRb/ObF6eOx8d9t koVSO7bG9foID/DUm29cGm7ne11yA06EmYyxjJQ0e8TD6ixxIPyiQIOr1cfvRT0/ e3mkrH+xJUByXmRu9elsOzDYk1xqwIRY6I5KcB1g3/2NxOTjG8DL7n3WTcKbea0t g0QOlFv5jLwsxa5l0SDn2ONtV+QipS/ecvTdnH+bk7FvY5bX0UcEURfQcKiMK9Sl fozS+AeukrBi4W72iVFYeQxxaYpJyEqRp457yq++wPGbeJP8w0cn7Cke9a8i9M7h pudRazu6MXF1u9c2kWscCugdpHAcEA+HXS94Ah5RlXkX9VBqKoc2pjrzx+wrZcCh nAODUWdFx8C654hmjFSD0oUKNaG5WEwJbQaAwFCrOdlB31Teeqw10e3ArcI/D9OL dnWFXGdD/QKKoCfgMCs3nFhPDmGuMzcBDopR5bxF5DDspoALxiHbLS6eRIXhYH+i aieeERi4GnJsCRfCVJbqK4qOq2ePuwi8HDptRNTv8HiHmC4uNz51k/VL0cqHpjlw 8ZrSUFAe4h6D7TsrLJ9ehac8J1QlAtycq1MX/4fleWXf2dK53kgNBbPDNBQsV2lZ X34Ooo+zOV15p7ijQ2en =mdwF -----END PGP SIGNATURE-----

2 3

how to override the LogbackMDCAdapter
by Matt Young 11 Oct '13

11 Oct '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I am having trouble overriding the LogbackMDCAdapter. No matter what happens, MDC.getAdapter().getClass() returns ch.qos.logback.classic.util.LogbackMDCAdapter It is important to mention that I am using slf4j with logback. Here is my config: <?xml version="1.0" encoding="UTF-8"?> <configuration> <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">  <encoder> <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%thread] %C:%L %msg%n</pattern> </encoder> </appender> <appender name="jobLoggerAppender" class="ch.qos.logback.classic.sift.SiftingAppender"> <discriminator class="ch.qos.logback.classic.sift.MDCBasedDiscriminator"> <key>jobGroup</key> <key>jobName</key> <key>jobId</key> <defaultValue>unknown</defaultValue> </discriminator> <sift> <appender name="${jobGroup}-${jobName}-${jobId}" class="ch.qos.logback.core.FileAppender"> <file>${jobGroup}/${jobName}/${jobId}.log</file> <layout class="ch.qos.logback.classic.PatternLayout"> <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%thread] %C:%L %msg%n</pattern> </layout> </appender> </sift> </appender>   <root level="DEBUG"> <appender-ref ref="STDOUT" /> <appender-ref ref="FILE" /> </root> <logger name="jobLogger" level="debug" > <appender-ref ref="jobLoggerAppender"/> </logger> </configuration> As you can see, I do have the class attribute set on the decriminator. And I use the slf4j MDC class to put() the 3 values jobGroup, jobName and jobId in the MDC. Any ideas? - -- Cheers, Matt Young solid(a)youngdev.net http://youngdev.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSV2oCAAoJEO5jycTTPEzc9WsP/3LhDRkAcSrmj3ErxfhqfY1b s4+irKclxjHzfYItoxDKc0LhPPfm4QM0aU17gwbR4a6hjBK5coMEWYRRZsB94OuO g3XsnWjvELbuaexXqpAm0Gep7iqBUm57/gi96Jb0SJZSCoInn1b0oMBzydKWSgJe lU9YLgoI+m4Mk6o+qPdrMtZvErNeLuleMru45g5bAl/KvDpCnYP5t3ErKqOukkUz 1rJsE6mx8e8g/VVtQXFk77gmSkuoVXynOuBI5c6NvQEbtoBawqVckvI9HcYHqIE6 7w+PXmIn4r6l9vs9KYz4kNQ2wBL007vGTDfyx2ZC/AhoqgFXNX/PDGC8rIWhMnG8 BCGqs+4EjKhazzpT9XSv7fU99SQ0e0qPhGfa7g/MDfPMsZWjKChoiRMnHEKD4YHr xJQm6YxnlBpyxg8+LNKLfwdBNNiHNa47uw+BaFQNslAG0FC+/qM+7M8kxa8KjkAW zU/Hsi+ZCpECaMZ5w2v2Hn+4ZIoKPG+wUrEYH600TM9isanLeNfDXlVPTfvozSMu VEm5y/wmPoZiAZfP0Dkmkx6sMHR48ccYdV+o0q7R8N4NwnETiTbITS/xE/nkSNwh YgDdG8wvv/yJaByNGBRAXD7NCT8wCwCTzZpglfTZjA8NdXebsctMgGCrINnRuYLl 5BtLgxc5DzBVaYdB+jZ5 =A65C -----END PGP SIGNATURE-----

1 0

Is there a non syslog UDP appender?
by Adam Gordon 09 Oct '13

09 Oct '13

Hey all, does there exist already a UDP appender for remote logging that is not syslog? If not, could the SocketAppender be adapted to do so? Thanks, --adam http://gordonizer.com

1 0

Log Rotation/Gzip process leaving .tmp files behind
by ofer lilian 07 Oct '13

07 Oct '13

http://jira.qos.ch/browse/LOGBACK-173 is there any progress with this bug fix ?

3 4

Custom Appender Substitute Logger issue
by Jared Graber 07 Oct '13

07 Oct '13

I have a custom appender that needs a backup logger in case there is a network issue or some other common error (see sample code below). I'm running into the error indicating that my backup logger will not work because it was created during the default configuration phase. It directed me towards the docs on substituteLogger which suggested to set up my appender in a separate file and then configure it after the default configuration phase. Are there any other options? Is there a better way to deal w/ the backup logger? Is there a more general purpose way to do this? Maybe a composite appender? Any help would be appreciated. My class basically looks like this: public class FooAppender extends AppenderBase<ILoggingEvent> { private final Logger LOGGER = LoggerFactory.getLogger(FooAppender.class); private Layout<ILoggingEvent> layout; //A bunch of configuration and constants. //Some setter methods @Override Public void append(final ILoggingEvent event) { String message = this.layout.doLayout(event); //The thrift client will encode it as UTF-8 for us. try { //log the event } catch (SpecificException e) { LOGGER.error(message); } } } Sample logback.xml snippet: <appender name="foo" class="FooAppender"> <layout> <pattern>%m%n</pattern> <outputPatternAsPresentationHeader>false</outputPatternAsPresentationHeader> </layout> <!-more configuration --> </appender> <appender name="fooBackup" class="ch.qos.logback.core.FileAppender"> <file>/tmp/logs/fooBackup.log</file> <encoder> <charset>UTF-8</charset> <pattern>%m</pattern> <outputPatternAsPresentationHeader>false</outputPatternAsPresentationHeader> </encoder> </appender> <logger name="FooAppenderTester" level="INFO" additivity="false"> <appender-ref ref="foo"/> </logger> <logger name="FooAppender" level="INFO" additivity="false"> <appender-ref ref="fooBackup"/> </logger>

1 0

Logback with logrotate best practices
by Nino Walker 04 Oct '13

04 Oct '13

I've posted a question on this matter on stack overflow - http://stackoverflow.com/questions/19105608/using-logrotate-with-logback Can anyone here provide some further guidance? Cheers, Nino

1 0

logging multiple main programs
by Palmer, Eric 02 Oct '13

02 Oct '13

Hey, I have been using logback for some time with monolithic java batch apps (ETL). I am a very part time jave programmer at my job (10% at most) I now am working on an application that has two main programs but uses 90% of the same code base. So one project, two jar file artifacts. (two programs labeled people and position) The logging is pretty simple * one log file for each run * log files are denoted with people_timestamp.log or position_timestamp.log * error logging sends email as well as writes to the log file this is my problem: When I run the first main program that is working it creates both log files and one is empty. I only want the one log file that is needed created. I'm sure I'm making this way more complicated than it needs to be. In the code base for people PeopleXmlMain.log = LoggerFactory.getLogger("people"); PeopleXmlMain.logErr = LoggerFactory.getLogger("peopleerror"); similar for position main PositionXmlMain.log = LoggerFactory.getLogger("position"); PositionXmlMain.logErr = LoggerFactory.getLogger("positionerror"); the config file ================================ <configuration debug="true"> <timestamp key="bySecond" datePattern="yyyyMMdd'T'HHmmss"/>  <appender name="ERROREMAIL" class="ch.qos.logback.classic.net.SMTPAppender"> <cyclicBufferTracker class="ch.qos.logback.core.spi.CyclicBufferTracker">  <bufferSize>1</bufferSize> </cyclicBufferTracker> <smtpHost>autosmtp.richmond.edu</smtpHost> <to>someone(a)somedomain.com</to>  <subject>Error in directory loader: %logger{20}</subject> <layout class="ch.qos.logback.classic.PatternLayout"> <pattern>%date %-30(%d{HH:mm:ss.SSS} [%M]) %-5level %-33(%C{32}) - %msg%n%n%ex%n%n</pattern> </layout> </appender> <appender name="ERROREMAILPEOPLE" class="ch.qos.logback.classic.net.SMTPAppender"> <cyclicBufferTracker class="ch.qos.logback.core.spi.CyclicBufferTracker">  <bufferSize>1</bufferSize> </cyclicBufferTracker> <smtpHost>autosmtp.richmond.edu</smtpHost> <to>someone(a)somedomain.com</to>  <subject>Error in People2Xml: %logger{20}</subject> <layout class="ch.qos.logback.classic.PatternLayout"> <pattern>%date %-30(%d{HH:mm:ss.SSS} [%M]) %-5level %-33(%C{32}) - %msg%n%n%ex%n%n</pattern> </layout> </appender> <appender name="ERROREMAILPOSITION" class="ch.qos.logback.classic.net.SMTPAppender"> <cyclicBufferTracker class="ch.qos.logback.core.spi.CyclicBufferTracker"> <bufferSize>1</bufferSize> </cyclicBufferTracker> <smtpHost>autosmtp.richmond.edu</smtpHost> <to>someone(a)somedomain.com</to> <subject>Error in Position2Xml: %logger{20}</subject> <layout class="ch.qos.logback.classic.PatternLayout"> <pattern>%date %-30(%d{HH:mm:ss.SSS} [%M]) %-5level %-33(%C{32}) - %msg%n%n%ex%n%n</pattern> </layout> </appender> <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">  <encoder> <pattern>%-40(%d{HH:mm:ss.SSS} [%M]) %-5level %-36(%C{32}) - %msg%n</pattern> </encoder> </appender> <appender name="FILEPEOPLE" class="ch.qos.logback.core.FileAppender"> <file>/somepath/logs/people_${bySecond}.log</file> <append>true</append>  <encoder> <pattern>%-40(%d{HH:mm:ss.SSS} [%M]) %-5level %-36(%C{32}) - %msg%n</pattern> </encoder> </appender> <appender name="FILEPOSITION" class="ch.qos.logback.core.FileAppender"> <file>/somepath/logs/position_${bySecond}.log</file> <append>true</append>  <encoder> <pattern>%-40(%d{HH:mm:ss.SSS} [%M]) %-5level %-36(%C{32}) - %msg%n</pattern> </encoder> </appender>  <logger name="org.apache" level="warn" /> <logger name="Error" level="error"> <appender-ref ref="ERROREMAIL" /> </logger> <logger name="people" level="info"> <appender-ref ref="FILEPEOPLE" /> <appender-ref ref="ERROREMAIL" /> </logger> <logger name="position" level="info"> <appender-ref ref="FILEPOSITION" /> <appender-ref ref="ERROREMAIL" /> </logger> <logger name="peopleerror" level="info"> <appender-ref ref="ERROREMAILPEOPLE" /> </logger> <logger name="positionerror" level="info"> <appender-ref ref="ERROREMAILPOSITION" /> </logger> <root level="info"> <appender-ref ref="STDOUT" /> </root> </configuration> -- Eric Palmer Web Services U of Richmond To report technical issues, obtain technical support or make requests for enhancements please visit http://web.richmond.edu/contact/technical-support.html

1 0

Fwd: Managing Logging JMXConfiguration via weblogic.WLST
by Troy Hart 02 Oct '13

02 Oct '13

Help please. I am deploying my application in weblogic server 12. I have my application configured with the jmxConfigurator. In my development environment I have disabled jmx remote authentication with the following java args: -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9999 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false With this configuration I can connect to a running weblogic process using jconsole. Within this tool there is an MBeans tab and from there I can navigate to a node named "ch.qos.logback.classic" and I can see and modify my application's logging configuration. This works great for development. When I go to production I want to provide a way for the CM/Support team to administer the configuration at runtime. However, I can not disable jmx remote authentication for this scenario. Also, I don't think I'll be able to use jconsole in this environment. I will be able to use weblogic's WLST tool though. The problem is that I can't find or configure the MBeans within the WLST tool. Attached is a screen shot of jconsole, within which I can modify the configuration. Then below, I will show some of my failing in wlst: > wls:/emall/domainConfig> find(name="ch.qos.logback.classic") > Finding 'ch.qos.logback.classic' in all registered MBean instances ... > No results found in MBean Instances > > wls:/emall/domainConfig> find(name="acct") > Finding 'acct' in all registered MBean instances ... > No results found in MBean Instances > > wls:/emall/domainConfig> > find(name="ch.qos.logback.classic.jmx.JMXConfigurator") > Finding 'ch.qos.logback.classic.jmx.JMXConfigurator' in all registered > MBean instances ... > No results found in MBean Instances > > wls:/emall/domainConfig> > find(name="ch.qos.logback.classic:Name=acct,Type=ch.qos.logback.classic.jmx.JMXConfigurator") > Finding > 'ch.qos.logback.classic:Name=acct,Type=ch.qos.logback.classic.jmx.JMXConfigurator' > in all registered MBean instances ... > No results found in MBean Instances > > wls:/emall/serverRuntime> serverConfig() > > wls:/emall/serverConfig> find(name="ch.qos.logback.classic") > Finding 'ch.qos.logback.classic' in all registered MBean instances ... > No results found in MBean Instances > > wls:/emall/serverConfig> find(name="acct") > Finding 'acct' in all registered MBean instances ... > No results found in MBean Instances > > wls:/emall/serverConfig> > find(name="ch.qos.logback.classic.jmx.JMXConfigurator") > Finding 'ch.qos.logback.classic.jmx.JMXConfigurator' in all registered > MBean instances ... > No results found in MBean Instances > > wls:/emall/serverConfig> > find(name="ch.qos.logback.classic:Name=acct,Type=ch.qos.logback.classic.jmx.JMXConfigurator") > Finding > 'ch.qos.logback.classic:Name=acct,Type=ch.qos.logback.classic.jmx.JMXConfigurator' > in all registered MBean instances ... > No results found in MBean Instances [image: Inline image 1]

1 0

Re: [logback-user] Multiple log files to single archive
by PaSik 02 Oct '13

02 Oct '13

Well, we have configured a rolling appender by date and size and have an app that produces lots of logs - hundreds of 40MB files. We have a script that compresses those files at the end of the day but we have some problems with it - sometimes files are locked by logback, there's also the topic of multiple environments. It'd be nice if the logback itself did it for us, but I don't want to end up with hundreds od zip files instead. Anyway, thanks for your insight. If I do decide to implement something and it works I'll then create a ticket with the contribution. Regards, Tom David Roussel <nabble(a)diroussel.xsmail.com> wrote: >Tom, > >If you create a ticket no one is going to do the work for you as I don't think anyone else needs this functionality. If you implement it by all means pleas contribute back. I'm not a commiter, just a user. > >You could start by subclassing one of the existing rolling implementations and add in your extra logic. > >It does all seem to get complicated quickly. How do you configure the archive interval? A new tag in the confg? How do you match all the older files, but not the current one etc. > >What advantage do you think you are gaining from this scheme you are suggesting? > >David > >> On 2 Oct 2013, at 08:35, PaSik <tompasik(a)poczta.fm> wrote: >> >> I wasn't aware that Java doesn't provide any API to modify ZIP archives, that's a shake :-( >> Do you think it would be doable if the request would be to create the zip archive once at the time the archive should roll. I guess the concept of rolling should be twofold - the log files roll independently of the archives and the archives roll every n archive rolls. The moment the archive rolls all logs should be archived. >> In case of %d{yyyy-MM-dd}.%i.zip pattern on every change of %i the next log would be created (no archiving happens at this time) and on the change of the date the rolling should happen as usual but also all the %i logs created on the previous date should be archived. >> Do you think I can create a ticket for such a feature? >> >> Regards, >> Tom >> >> David Roussel <nabble(a)diroussel.xsmail.com> wrote: >> >>> No that is not supported. >>> >>> Note that there are no java APIs (that I know of) for modifying a zip file. The only way to add a file to an existing zip file is to create a new zip file with the file added and then delete the old zip file. >>> >>> David >>> >>>> On 1 Oct 2013, at 09:57, PaSik <tompasik(a)poczta.fm> wrote: >>>> >>>> Hi, >>>> >>>> Could you please tell me whether it's possible to configure logback in such a way that multiple log files are compressed to a single, possibly already existing archive (existing because logback itself has already put some log Giles in this archive). >>>> >>>> The current use case is that I'd like to have a RollingFileAppender with SizeAndTimeBasedFNATP with fileNamePattern=mylog-%d{yyyy-MM-dd}.%i.zip and have the log files archived to mylog-%d{yyyy-MM-dd}.zip. >>>> >>>> Another use case would be an app that doesn't produce much logs and the loga would be produced to daily files but the archives would be monthly. >>>> >>>> Thanks, >>>> Tom >>>> _______________________________________________ >>>> Logback-user mailing list >>>> Logback-user(a)qos.ch >>>> http://mailman.qos.ch/mailman/listinfo/logback-user >>> _______________________________________________ >>> Logback-user mailing list >>> Logback-user(a)qos.ch >>> http://mailman.qos.ch/mailman/listinfo/logback-user >> _______________________________________________ >> Logback-user mailing list >> Logback-user(a)qos.ch >> http://mailman.qos.ch/mailman/listinfo/logback-user >_______________________________________________ >Logback-user mailing list >Logback-user(a)qos.ch >http://mailman.qos.ch/mailman/listinfo/logback-user

1 0

Re: [logback-user] Multiple log files to single archive
by PaSik 02 Oct '13

02 Oct '13

I wasn't aware that Java doesn't provide any API to modify ZIP archives, that's a shake :-( Do you think it would be doable if the request would be to create the zip archive once at the time the archive should roll. I guess the concept of rolling should be twofold - the log files roll independently of the archives and the archives roll every n archive rolls. The moment the archive rolls all logs should be archived. In case of %d{yyyy-MM-dd}.%i.zip pattern on every change of %i the next log would be created (no archiving happens at this time) and on the change of the date the rolling should happen as usual but also all the %i logs created on the previous date should be archived. Do you think I can create a ticket for such a feature? Regards, Tom David Roussel <nabble(a)diroussel.xsmail.com> wrote: >No that is not supported. > >Note that there are no java APIs (that I know of) for modifying a zip file. The only way to add a file to an existing zip file is to create a new zip file with the file added and then delete the old zip file. > >David > >> On 1 Oct 2013, at 09:57, PaSik <tompasik(a)poczta.fm> wrote: >> >> Hi, >> >> Could you please tell me whether it's possible to configure logback in such a way that multiple log files are compressed to a single, possibly already existing archive (existing because logback itself has already put some log Giles in this archive). >> >> The current use case is that I'd like to have a RollingFileAppender with SizeAndTimeBasedFNATP with fileNamePattern=mylog-%d{yyyy-MM-dd}.%i.zip and have the log files archived to mylog-%d{yyyy-MM-dd}.zip. >> >> Another use case would be an app that doesn't produce much logs and the loga would be produced to daily files but the archives would be monthly. >> >> Thanks, >> Tom >> _______________________________________________ >> Logback-user mailing list >> Logback-user(a)qos.ch >> http://mailman.qos.ch/mailman/listinfo/logback-user >_______________________________________________ >Logback-user mailing list >Logback-user(a)qos.ch >http://mailman.qos.ch/mailman/listinfo/logback-user

2 1