- logback-user - mailman3.qos.ch

FixedWindowRollingPolicy with date rename on roll-over
by Deepak Abraham 20 Aug '19

20 Aug '19

Hello, I am able to use the FixedWindowRollingPolicy along with SizeBasedTriggeringPolicy to achieve roll over based on defined file size (logback-1.2.3) . However I need to change the name of the file on roll-over such that instead of the index the current date and time would be used in the rolled over file name. For example: if the current file name is abcApp.log, the rolled over file name should be abcApp_2019-08-20_203522.log (The roll0over would happen when the defined file size is reached based on the defined SizeBasedTriggeringPolicy) Is there any way in existing rolling Policy classes that support this? I couldn't find any. Or any other way to achieve this? If there is no support for this, I would be willing to add support by either tweaking the existing FixedWindowRollingPolicy class or creating a similar new class and supplying the same. Please let me know. Thanks & Regards, Deepak.

1 0

Implementing MDC with ServerSocketReceiver
by Joaquim Jose 30 Jul '19

30 Jul '19

Hi, I'm trying to implement a TCP server which logs to different appenders depending on the message content. I'm using ServerSocketReceiver and SiftingAppender strategy based on MDC value. Here's my snippet: siftingAppender.setAppenderFactory((Context context1, String discriminatingValue) -> { Appender appender; if (DISCRIMINATOR_DEFAULT_VALUE.equals(discriminatingValue)) { ConsoleAppender<ILoggingEvent> consoleAppender = new ConsoleAppender<>(); consoleAppender.setContext(context); consoleAppender.setEncoder(consoleOnlyEncoder(context)); consoleAppender.setName("MY_CONSOLE_APPENDER"); consoleAppender.start(); appender = consoleAppender; } else { RollingFileAppender rollingFileAppender = rollingFileAppender("MY_ROLLING_APPENDER", "myPrefix." + discriminatingValue + ".%d{yyyy-MM-dd_HH-mm}.log", context); AsyncAppender asyncAppender = asyncAppender(rollingFileAppender, context); appender = asyncAppender; } return appender; }); Let's say that my MDC discriminator key is "filename". My first challenge with this approach is that the MDC value is not serialized and sent to server. So, I changed the message format to "filename={},msg{}" and I created a Turbo Filter, which would check if the rawMessage is equal to this expression and, if so, It would log using the MDC discriminator. Here's the snippet: private static void addMDCInterceptorTurboFilter(LoggerContext context) { String regex = DISCRIMINATOR_KEY.concat("={},").concat(MESSAGE_KEY).concat("={}"); // "filename={},msg={}" TurboFilter mdcInterceptorFilter = new TurboFilter() { @Override public FilterReply decide(Marker marker, ch.qos.logback.classic.Logger logger, Level level, String rawMessage, Object[] params, Throwable t) { if (params != null && regex.equals(rawMessage)) { MDC.put(DISCRIMINATOR_KEY, (String) params[0]); logger.log(marker, this.getClass().getName(), Level.toLocationAwareLoggerInteger(level), (String) params[1], null, null); MDC.remove(DISCRIMINATOR_KEY); return FilterReply.DENY; } return FilterReply.NEUTRAL; } }; context.addTurboFilter(mdcInterceptorFilter); mdcInterceptorFilter.setName("MY_MDC_INTERCEPTOR_FILTER"); mdcInterceptorFilter.start(); } My challenge in the above approach is that neither the *rawMessage* nor *params* are populated at this time. So I can't set the MDC properly. Another approach that I tried was to set the filter to the SiftingAppender and set the MDC there, but it didn't work either. Here's the snippet: private Filter<ILoggingEvent> addMDCInterceptorFilter(String keyDiscriminator, String messageKey) { String regex = keyDiscriminator.concat("={},").concat(messageKey).concat("={}"); Filter<ILoggingEvent> filter = new Filter<ILoggingEvent>() { @Override public FilterReply decide(ILoggingEvent event) { if (regex.equals(event.getMessage())) { MDC.put(keyDiscriminator, (String)event.getArgumentArray()[0]); } else { MDC.remove(keyDiscriminator); } return FilterReply.NEUTRAL; } }; filter.start(); return filter; } Could you guys guide me how to accomplish this scenario ? Please find the full source for sake of completeness. Thanks, import ch.qos.logback.classic.AsyncAppender; import ch.qos.logback.classic.Level; import ch.qos.logback.classic.LoggerContext; import ch.qos.logback.classic.encoder.PatternLayoutEncoder; import ch.qos.logback.classic.net.server.ServerSocketReceiver; import ch.qos.logback.classic.sift.MDCBasedDiscriminator; import ch.qos.logback.classic.sift.SiftingAppender; import ch.qos.logback.classic.spi.ILoggingEvent; import ch.qos.logback.classic.turbo.TurboFilter; import ch.qos.logback.core.Appender; import ch.qos.logback.core.ConsoleAppender; import ch.qos.logback.core.Context; import ch.qos.logback.core.encoder.Encoder; import ch.qos.logback.core.rolling.RollingFileAppender; import ch.qos.logback.core.rolling.RollingPolicy; import ch.qos.logback.core.rolling.TimeBasedRollingPolicy; import ch.qos.logback.core.spi.FilterReply; import ch.qos.logback.core.util.StatusPrinter; import org.slf4j.LoggerFactory; import org.slf4j.MDC; import org.slf4j.Marker; import java.nio.charset.Charset; public class MySocketServer { private static final String DISCRIMINATOR_KEY = "filename"; private static final String DISCRIMINATOR_DEFAULT_VALUE = "unknown"; private static final String MESSAGE_KEY = "msg"; public static void main(String[] args) throws InterruptedException { LoggerContext context = (LoggerContext) LoggerFactory.getILoggerFactory(); StatusPrinter.print(context); addMDCInterceptorTurboFilter(context); newServerSocketReceiver(context); addAppenders(context); Thread.sleep(Long.MAX_VALUE); } private static void newServerSocketReceiver(LoggerContext context) { ServerSocketReceiver serverSocketReceiver = new ServerSocketReceiver(); serverSocketReceiver.setContext(context); serverSocketReceiver.setPort(6000); serverSocketReceiver.setAddress("localhost"); serverSocketReceiver.start(); } private static void addAppenders(LoggerContext context) { SiftingAppender siftingAppender = siftingAppender(context); ch.qos.logback.classic.Logger logger = context.getLogger(ch.qos.logback.classic.Logger.ROOT_LOGGER_NAME); logger.detachAndStopAllAppenders(); logger.addAppender(siftingAppender); } private static SiftingAppender siftingAppender(LoggerContext context) { SiftingAppender siftingAppender = new SiftingAppender(); siftingAppender.setContext(context); siftingAppender.setName("MY_SIFT_APPENDER"); siftingAppender.setDiscriminator(getMdcBasedDiscriminator(context)); siftingAppender.setAppenderFactory((Context context1, String discriminatingValue) -> { Appender appender; if (DISCRIMINATOR_DEFAULT_VALUE.equals(discriminatingValue)) { ConsoleAppender<ILoggingEvent> consoleAppender = new ConsoleAppender<>(); consoleAppender.setContext(context); consoleAppender.setEncoder(consoleOnlyEncoder(context)); consoleAppender.setName("MY_CONSOLE_APPENDER"); consoleAppender.start(); appender = consoleAppender; } else { RollingFileAppender rollingFileAppender = rollingFileAppender("MY_ROLLING_APPENDER", "myPrefix." + discriminatingValue + ".%d{yyyy-MM-dd_HH-mm}.log", context); AsyncAppender asyncAppender = asyncAppender(rollingFileAppender, context); appender = asyncAppender; } return appender; }); siftingAppender.start(); return siftingAppender; } private static AsyncAppender asyncAppender(Appender appender, LoggerContext context) { AsyncAppender asyncAppender = new AsyncAppender(); asyncAppender.setContext(context); asyncAppender.setName("MY_ASYNC_BACKUP_APPENDER"); asyncAppender.setQueueSize(512); asyncAppender.addAppender(appender); asyncAppender.start(); return asyncAppender; } private static MDCBasedDiscriminator getMdcBasedDiscriminator(LoggerContext context) { MDCBasedDiscriminator discriminator = new MDCBasedDiscriminator(); discriminator.setContext(context); discriminator.setKey(DISCRIMINATOR_KEY); discriminator.setDefaultValue(DISCRIMINATOR_DEFAULT_VALUE); discriminator.start(); return discriminator; } private static RollingFileAppender rollingFileAppender(String name, String fileNamePattern, LoggerContext context) { RollingFileAppender rollingFileAppender = new RollingFileAppender(); rollingFileAppender.setContext(context); rollingFileAppender.setRollingPolicy(timeBasedRollingPolicy(rollingFileAppender, fileNamePattern, context)); rollingFileAppender.setEncoder(messageOnlyEncoder(context)); rollingFileAppender.setName(name); rollingFileAppender.start(); return rollingFileAppender; } private static RollingPolicy timeBasedRollingPolicy(RollingFileAppender rollingFileAppender, String fileNamePattern, LoggerContext context) { final TimeBasedRollingPolicy timeBasedRollingPolicy = new TimeBasedRollingPolicy(); timeBasedRollingPolicy.setContext(context); timeBasedRollingPolicy.setFileNamePattern(fileNamePattern); timeBasedRollingPolicy.setParent(rollingFileAppender); timeBasedRollingPolicy.start(); return timeBasedRollingPolicy; } private static Encoder messageOnlyEncoder(LoggerContext context) { final PatternLayoutEncoder patternLayoutEncoder = new PatternLayoutEncoder(); patternLayoutEncoder.setPattern("%mdc %msg%n"); patternLayoutEncoder.setCharset(Charset.forName("utf-8")); patternLayoutEncoder.setContext(context); patternLayoutEncoder.start(); return patternLayoutEncoder; } private static Encoder consoleOnlyEncoder(LoggerContext context) { final PatternLayoutEncoder patternLayoutEncoder = new PatternLayoutEncoder(); patternLayoutEncoder.setPattern("%-4relative [%thread] %-5level %logger{35} - %msg %n"); patternLayoutEncoder.setCharset(Charset.forName("utf-8")); patternLayoutEncoder.setContext(context); patternLayoutEncoder.start(); return patternLayoutEncoder; } private static void addMDCInterceptorTurboFilter(LoggerContext context) { String regex = DISCRIMINATOR_KEY.concat("={},").concat(MESSAGE_KEY).concat("={}"); // "filename={},msg={}" TurboFilter mdcInterceptorFilter = new TurboFilter() { @Override public FilterReply decide(Marker marker, ch.qos.logback.classic.Logger logger, Level level, String rawMessage, Object[] params, Throwable t) { if (params != null && regex.equals(rawMessage)) { MDC.put(DISCRIMINATOR_KEY, (String) params[0]); logger.log(marker, this.getClass().getName(), Level.toLocationAwareLoggerInteger(level), (String) params[1], null, null); MDC.remove(DISCRIMINATOR_KEY); return FilterReply.DENY; } return FilterReply.NEUTRAL; } }; context.addTurboFilter(mdcInterceptorFilter); mdcInterceptorFilter.setName("MY_MDC_INTERCEPTOR_FILTER"); mdcInterceptorFilter.start(); } }

1 0

SMTPAppender and JaninoEventEvaluator not working as expected
by Thorsten Schöning 09 Jul '19

09 Jul '19

Hi all, I have an SMTPAppender in place which should send mails in case of log level WARN as well instead of only ERROR. That works properly with the following expression and a custom implementation checking the log level: > <evaluator class="de.am_soft.util.logging.logback.OnWarnEvaluator" /> I have an additional SMTPAppender which should recognize the WARN-check as well, but check additional aspects of the logger and the log message. I thought that using an ADDITIONAL filter is the correct thing to do: > <evaluator class="de.am_soft.util.logging.logback.OnWarnEvaluator" /> > <filter class="ch.qos.logback.core.filter.EvaluatorFilter"> > <evaluator class="ch.qos.logback.classic.boolex.JaninoEventEvaluator"> > <expression> > return logger.endsWith(".emptyPacket") || > ((throwableProxy != null) && > throwableProxy.getMessage().contains("missing collectors")); > </expression> > </evaluator> > > <onMatch>ACCEPT</onMatch> > <onMismatch>DENY</onMismatch> > </filter> But that doesn't work, the receiver gets ALL mails with log level WARN, the additional filter doesn't seem to be applied. So I simply removed OnWarnEvaluator because I thought that the filter itself should be enough to decide, but that doesn't work as well. The receiver gets only mails with log level ERROR now. So how do OnWarnEvaluator and filter interact? Which one gets executed first or even at all? Regarding my tests it seems that only OnWarnEvaluator gets executed. Does the filter need to check the log level as well? Wouldn't make much sense to me, because the appender doesn't care about which things are checked within the filter, only about its results. Thanks for any clarifications! Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning E-Mail: Thorsten.Schoening(a)AM-SoFT.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...........05151- 9468- 55 Fax...............05151- 9468- 88 Mobil..............0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow

1 0

How to force status listener output into the configured logfile
by Norbert Kiesel 27 Jun '19

27 Jun '19

Hi, I use logback for a component running in Tomcat. Currently, I have the logback status written to stdout (and redirected into a tomcat-stdout.log file). All my other logs are written to a file configured in logback.xml. How can I get the logback status written to that file (ideally just there and not to stdout, but I could also live with that info written twice). I'm aware of <https://logback.qos.ch/apidocs/ch/qos/logback/core/status/NopStatusListener…> OnPrintStreamStatusListenerBase<https://logback.qos.ch/apidocs/ch/qos/logback/core/status/OnPrintStreamStat…> and StatusListenerAsList<https://logback.qos.ch/apidocs/ch/qos/logback/core/status/StatusListenerAsL…> but they seem to be no help for this. </nk> --- Norbert Kiesel Systems Architect, Engineering E: nkiesel(a)metricstream.com<mailto:sonia.sharma@metricstream.com> W: www.metricstream.com

1 0

logger name too long
by Joud Kassem 29 May '19

29 May '19

Hi all, I'm new to logback and I have the following question: Is there a way to show only the left part of a logger name? like if I have "liquibase.changelog.ChangeSet" logger, I need to show only liquibase in the log. In the log pattern the focus always on the right part not the left as I see (%logger{36}). If not, is there a way to rename a logger in a plugin? like in the liquibase-core plugin, I want liquibase.changelog.ChangeSet to be liquibase.. Thanks in advance

1 0

Logs from Receiver not being displayed by Appenders
by ULStudent:ADAM.O'MAHONY 29 May '19

29 May '19

Hi all, Recently I have been trying to implement a centralised loggin server using slf4j with Logback. To try and understand how the receiver architecture works, I set up two java applications built with maven. One (Client) creates logs and sends them via ch.qos.logback.classic.net.SocketAppender to the second (Server) receiving with a ch.qos.logback.classic.net.server.ServerSocketReceiver. The problem I have encountered is that the logs sent from the client are not displayed by the Server's appenders. I have used a packet sniffer to confirm that the packets are indeed send and received. I have also tried changing the receiver and socket layout i.e SocketAppender -> ServerSocketAppender and ServerSocketReceiver -> SocketReceiver but got the same results. When nothing I was trying worked I tried using the examples given in the Logback documentation exactly but again, got the same result. Finally I tried sending logs via Log4j 2 (Log4j 2 Socket Appender -> Logback ServerSocketReceiver) but again no luck. The connect was established between appender and receiver in each test before logs began to send. I would like to also note that any logs local to the applications are displayed perfectly by their respected appenders. Any help with this problem would be greatly appreciated. Kind regards, Adam.

1 0

<logger/> to spi
by Rolf Schumacher 25 May '19

25 May '19

Just made my first spi.Configurator service work and translated some logback.xml to Java. Now I wonder what would be the best way to translate a <logger/> element, defining Level for sets of loggers? Should I define a Filter and add it to all Appenders, or is there a more straight forward way? Kind Regards Rolf

1 0

Re: [logback-user] No logfile when starting application from a remote host : Problem solved
by Michael.Fankanowsky＠t-systems.com 23 May '19

23 May '19

Hi, I could solve the problem. I didn’t consider that after login via ssh on the remote host the ‘execution’ environment is the user’s home directory. But our application is in a different directory and we configured the logs to be there (we detected the logs by accident in the home directory). Thus I had to combine two commands in ssh call with ProxyCommand – first a change directory and then the call of the start script. Now it works. Sorry for the confusion. Best regards, Michael Fankanowsky Von: logback-user <logback-user-bounces(a)qos.ch> Im Auftrag von Oleksandr Gavenko Gesendet: Montag, 13. Mai 2019 15:53 An: logback users list <logback-user(a)qos.ch> Betreff: Re: [logback-user] No logfile when starting application from a remote host Spring Boot is a magic. They probably expose some JMX endpoints but I know a way with Actuator endpoints, check in that order of importance: curl http://localhost:8080/actuator/env curl http://localhost:8080/actuator/loggers curl http://localhost:8080/actuator/ configprops On Mon, May 13, 2019 at 12:34 PM <Michael.Fankanowsky(a)t-systems.com<mailto:Michael.Fankanowsky@t-systems.com>> wrote: Hi, we are using logback v1.2.3 in a Springboot application which is running on Linux. When starting the application locally on the Linux host, logfile is created and application logs to it. When starting the application from a remote host via ssh then no logfile is created. When restarting the application (logfile is present and application has been logging to it) from a remote host, then no log messages will be logged to the existing logfile. It is working when restarting the application locally on the Linux host. Does anyone have an idea concerning this behavior? Best regards, Michael Fankanowsky _______________________________________________ logback-user mailing list logback-user(a)qos.ch<mailto:logback-user@qos.ch> http://mailman.qos.ch/mailman/listinfo/logback-user

1 0

Why my logs contain null characters(holes)
by Tianjiao Liu 15 May '19

15 May '19

Hi all, I am using logback(SiftingAppender and RollingFileAppender) in my application. Whenever requests come in, they will be logged to different files according to the id from url query. This is working really well and I get the expected result. However, I noticed occasionally, there are some null characters in the middle of logs. I am not sure why this happens and how to reduce it. I did some research and the closest thing I found is https://serverfault.com/questions/71341/why-do-jboss-and-logrotate-create-l… They are using logrotate with log4j and that results in the null character. However, in my case, the whole log rotate and lifecycle is handled by logback. This is partial of my logback xml configuration: <appender name="SIFT" class="ch.qos.logback.classic.sift.SiftingAppender"><discriminator> <key>id</key> <defaultValue>main</defaultValue></discriminator><sift> <appender name="${id}Rolling" class="ch.qos.logback.core.rolling.RollingFileAppender"> <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy"> <fileNamePattern>${logRoot}/${id}.%d{"yyyy-MM-dd-HH", UTC}.%i.json</fileNamePattern> <maxFileSize>${logSizeLimit}</maxFileSize> <maxHistory>100</maxHistory> <totalSizeCap>20GB</totalSizeCap> </rollingPolicy> <encoder> <pattern>%msg%n</pattern> </encoder> </appender></sift> And in the java code: String jsonOutput = gson.toJson(ping); MDC.put("id", id)); logger.info(jsonOutput); MDC.remove("id"); The output log looks like : {"timestamp":{"seconds":1557048143,"nanos":028000000},"httpRequest":{"status":200,"remoteIp":"111.11.111.111","requestMethod":"GET","requestUrl":" https://my.example.com/?id=testid","userAgent":"Mozilla/5.0 "}} ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@{"timestamp":{"seconds":1557041043,"nanos":318000000},"httpRequest":{"status":200,"remoteIp":"111.11.111.111","requestMethod":"GET","requestUrl":" https://my.example.com/?id=testid","userAgent":"Mozilla/5.0 "}} I am using vim to open it, so it displays null character as ^@. These null characters are appended at the beginning of a real log message. I haven't found a pattern when this will happen. Does anyone have any idea why this happens? Best regards, TJ Liu

1 0

No logfile when starting application from a remote host
by Michael.Fankanowsky＠t-systems.com 13 May '19

13 May '19

Hi, we are using logback v1.2.3 in a Springboot application which is running on Linux. When starting the application locally on the Linux host, logfile is created and application logs to it. When starting the application from a remote host via ssh then no logfile is created. When restarting the application (logfile is present and application has been logging to it) from a remote host, then no log messages will be logged to the existing logfile. It is working when restarting the application locally on the Linux host. Does anyone have an idea concerning this behavior? Best regards, Michael Fankanowsky

2 1