Multiple FileAppenders writing to single file no longer working in Logback 1.2.3
Hello there Logback users. An application I’m responsible uses logback for its internal logging. It logs to various files at various levels. I’ve extended it successfully to reformat certain log messages and save them into a new log called “combined-audit.log”. This is all within a single JVM, logging to a file on a local filesystem. The reason for the three loggers/appenders is that I want to log from three different packages, at different levels, filter each differently, and encode each differently. This is all necessary because I cannot make changes to the app itself, so I’m trying to use Logback to accomplish my needs. The below configuration works in Logback 1.1.3 and not in Logback 1.2.3. In Logback 1.2.3, only messages from one of the three loggers are logged to the single file. When I change the file name to three different files, messages from each logger are in each file. I read through release notes from 1.1.3 to 1.2.3 and nothing jumped out at me as having the potential to change this behavior. Is this a supported configuration in 1.2.3? Does anyone have any ideas for how to accomplish the objective? <!-- Appends app audit messages to combined audit log --> <appender name="COMBINED_AUDIT" class="ch.qos.logback.core.FileAppender"> <File>${logging.directory}/combined-audit.log</File> <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> <charset>UTF-8</charset> <Pattern>%date{ISO8601}|%mdc{m.remote_addr}|%mdc{m.jsessionid}||%msg%n</Pattern> </encoder> </appender> <logger name="j.k.l" level="ALL" additivity="false"> <appender-ref ref="COMBINED_AUDIT"/> </logger> <!-- Appends LDAP auth messages to combined audit log --> <appender name="COMBINED_LDAP" class="ch.qos.logback.core.FileAppender"> <File>${logging.directory}/combined-audit.log</File> <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> <charset>UTF-8</charset> <Pattern>%date{ISO8601}|%mdc{m.remote_addr}|%mdc{m.jsessionid}|%replace(%replace(%replace(%msg){'^.*resultCode=',''}){',.*identifier=','|'}){', context=.*$',''}|LDAP|||%n</Pattern> </encoder> <filter class="ch.qos.logback.core.filter.EvaluatorFilter"> <evaluator> <expression>return message.contains("authenticate response=");</expression> </evaluator> <OnMatch>ACCEPT</OnMatch> <OnMismatch>DENY</OnMismatch> </filter> </appender> <logger name="a.b.c.d" level="DEBUG" additivity="false"> <appender-ref ref="COMBINED_LDAP"/> </logger> <!-- Appends LDAP invalid username messages to combined audit log --> <appender name="COMBINED_INVALID_USERNAME" class="ch.qos.logback.core.FileAppender"> <File>${logging.directory}/combined-audit.log</File> <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> <charset>UTF-8</charset> <Pattern>%date{ISO8601}|%mdc{m.remote_addr}|%mdc{m.jsessionid}|INVALID_USERNAME||LDAP|||%n</Pattern> </encoder> <filter class="ch.qos.logback.core.filter.EvaluatorFilter"> <evaluator> <expression>return message.contains("failed using filter=");</expression> </evaluator> <OnMatch>ACCEPT</OnMatch> <OnMismatch>DENY</OnMismatch> </filter> </appender> <logger name="v.w.x.y.z" level="INFO" additivity="false"> <appender-ref ref="COMBINED_INVALID_USERNAME"/> </logger>
Hello again Logback users, I’m “bumping” this thread as I’m still having to use Logback 1.1.3 for my application. I’d like a bug to be opened in Logback if anyone could confirm whether what I am trying to do “should work”. I do not know if the way I set it up in Logback 1.1.3 is “supported” or not. Please let me know, thank you so much! - John From: John Morton <jtmorton@calpoly.edu> Date: Wednesday, December 19, 2018 at 7:14 PM To: "logback-user@qos.ch" <logback-user@qos.ch> Subject: Multiple FileAppenders writing to single file no longer working in Logback 1.2.3 Hello there Logback users. An application I’m responsible uses logback for its internal logging. It logs to various files at various levels. I’ve extended it successfully to reformat certain log messages and save them into a new log called “combined-audit.log”. This is all within a single JVM, logging to a file on a local filesystem. The reason for the three loggers/appenders is that I want to log from three different packages, at different levels, filter each differently, and encode each differently. This is all necessary because I cannot make changes to the app itself, so I’m trying to use Logback to accomplish my needs. The below configuration works in Logback 1.1.3 and not in Logback 1.2.3. In Logback 1.2.3, only messages from one of the three loggers are logged to the single file. When I change the file name to three different files, messages from each logger are in each file. I read through release notes from 1.1.3 to 1.2.3 and nothing jumped out at me as having the potential to change this behavior. Is this a supported configuration in 1.2.3? Does anyone have any ideas for how to accomplish the objective? <!-- Appends app audit messages to combined audit log --> <appender name="COMBINED_AUDIT" class="ch.qos.logback.core.FileAppender"> <File>${logging.directory}/combined-audit.log</File> <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> <charset>UTF-8</charset> <Pattern>%date{ISO8601}|%mdc{m.remote_addr}|%mdc{m.jsessionid}||%msg%n</Pattern> </encoder> </appender> <logger name="j.k.l" level="ALL" additivity="false"> <appender-ref ref="COMBINED_AUDIT"/> </logger> <!-- Appends LDAP auth messages to combined audit log --> <appender name="COMBINED_LDAP" class="ch.qos.logback.core.FileAppender"> <File>${logging.directory}/combined-audit.log</File> <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> <charset>UTF-8</charset> <Pattern>%date{ISO8601}|%mdc{m.remote_addr}|%mdc{m.jsessionid}|%replace(%replace(%replace(%msg){'^.*resultCode=',''}){',.*identifier=','|'}){', context=.*$',''}|LDAP|||%n</Pattern> </encoder> <filter class="ch.qos.logback.core.filter.EvaluatorFilter"> <evaluator> <expression>return message.contains("authenticate response=");</expression> </evaluator> <OnMatch>ACCEPT</OnMatch> <OnMismatch>DENY</OnMismatch> </filter> </appender> <logger name="a.b.c.d" level="DEBUG" additivity="false"> <appender-ref ref="COMBINED_LDAP"/> </logger> <!-- Appends LDAP invalid username messages to combined audit log --> <appender name="COMBINED_INVALID_USERNAME" class="ch.qos.logback.core.FileAppender"> <File>${logging.directory}/combined-audit.log</File> <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> <charset>UTF-8</charset> <Pattern>%date{ISO8601}|%mdc{m.remote_addr}|%mdc{m.jsessionid}|INVALID_USERNAME||LDAP|||%n</Pattern> </encoder> <filter class="ch.qos.logback.core.filter.EvaluatorFilter"> <evaluator> <expression>return message.contains("failed using filter=");</expression> </evaluator> <OnMatch>ACCEPT</OnMatch> <OnMismatch>DENY</OnMismatch> </filter> </appender> <logger name="v.w.x.y.z" level="INFO" additivity="false"> <appender-ref ref="COMBINED_INVALID_USERNAME"/> </logger>
Hi John, Multiple appenders cannot write to the same file unless prudent mode is enabled. In the 1.2.x series, logback prevents the same file to be opened by different FileAppenders as this is usually a configuration error. The 1.3 does not have this check. I suggest that you use a single FileAppender but add a filter according to your needs. Best regards, -- Ceki On 23/09/2019 17:39, John Morton wrote:
Hello again Logback users,
I’m “bumping” this thread as I’m still having to use Logback 1.1.3 for my application. I’d like a bug to be opened in Logback if anyone could confirm whether what I am trying to do “should work”. I do not know if the way I set it up in Logback 1.1.3 is “supported” or not.
Please let me know, thank you so much!
- John
*From: *John Morton <jtmorton@calpoly.edu> *Date: *Wednesday, December 19, 2018 at 7:14 PM *To: *"logback-user@qos.ch" <logback-user@qos.ch> *Subject: *Multiple FileAppenders writing to single file no longer working in Logback 1.2.3
Hello there Logback users. An application I’m responsible uses logback for its internal logging. It logs to various files at various levels. I’ve extended it successfully to reformat certain log messages and save them into a new log called “combined-audit.log”. This is all within a single JVM, logging to a file on a local filesystem.
The reason for the three loggers/appenders is that I want to log from three different packages, at different levels, filter each differently, and encode each differently. This is all necessary because I cannot make changes to the app itself, so I’m trying to use Logback to accomplish my needs.
The below configuration works in Logback 1.1.3 and not in Logback 1.2.3.
In Logback 1.2.3, only messages from one of the three loggers are logged to the single file. When I change the file name to three different files, messages from each logger are in each file.
I read through release notes from 1.1.3 to 1.2.3 and nothing jumped out at me as having the potential to change this behavior.
Is this a supported configuration in 1.2.3? Does anyone have any ideas for how to accomplish the objective?
<!-- Appends app audit messages to combined audit log -->
<appender name="COMBINED_AUDIT" class="ch.qos.logback.core.FileAppender">
<File>${logging.directory}/combined-audit.log</File>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<charset>UTF-8</charset>
<Pattern>%date{ISO8601}|%mdc{m.remote_addr}|%mdc{m.jsessionid}||%msg%n</Pattern>
</encoder>
</appender>
<logger name="j.k.l" level="ALL" additivity="false">
<appender-ref ref="COMBINED_AUDIT"/>
</logger>
<!-- Appends LDAP auth messages to combined audit log -->
<appender name="COMBINED_LDAP" class="ch.qos.logback.core.FileAppender">
<File>${logging.directory}/combined-audit.log</File>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<charset>UTF-8</charset>
<Pattern>%date{ISO8601}|%mdc{m.remote_addr}|%mdc{m.jsessionid}|%replace(%replace(%replace(%msg){'^.*resultCode=',''}){',.*identifier=','|'}){', context=.*$',''}|LDAP|||%n</Pattern>
</encoder>
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<evaluator>
<expression>return message.contains("authenticate response=");</expression>
</evaluator>
<OnMatch>ACCEPT</OnMatch>
<OnMismatch>DENY</OnMismatch>
</filter>
</appender>
<logger name="a.b.c.d" level="DEBUG" additivity="false">
<appender-ref ref="COMBINED_LDAP"/>
</logger>
<!-- Appends LDAP invalid username messages to combined audit log -->
<appender name="COMBINED_INVALID_USERNAME" class="ch.qos.logback.core.FileAppender">
<File>${logging.directory}/combined-audit.log</File>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<charset>UTF-8</charset>
<Pattern>%date{ISO8601}|%mdc{m.remote_addr}|%mdc{m.jsessionid}|INVALID_USERNAME||LDAP|||%n</Pattern>
</encoder>
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<evaluator>
<expression>return message.contains("failed using filter=");</expression>
</evaluator>
<OnMatch>ACCEPT</OnMatch>
<OnMismatch>DENY</OnMismatch>
</filter>
</appender>
<logger name="v.w.x.y.z" level="INFO" additivity="false">
<appender-ref ref="COMBINED_INVALID_USERNAME"/>
</logger>
Ceki, I know how to have a single appender with a complex filter that would work to isolate the messages. The issue though is that I believe each appender can only have a single encoder. Due to the origin/content of the different types of messages, I'm using different encoders for each of the three types (in each of the three appenders). The only way I see being able to do this is with multiple appenders. Can you think of another way to do this with logback? I've tried prudent mode on in 1.2.3 and it does not seem to make a difference. Was your 1.3 comment meant to indicate that the behavior should be the same as in 1.1.3 that I relied on previously? - John On 9/24/19, 3:59 AM, "logback-user on behalf of Ceki Gülcü" <logback-user-bounces@qos.ch on behalf of ceki@qos.ch> wrote: Hi John, Multiple appenders cannot write to the same file unless prudent mode is enabled. In the 1.2.x series, logback prevents the same file to be opened by different FileAppenders as this is usually a configuration error. The 1.3 does not have this check. I suggest that you use a single FileAppender but add a filter according to your needs. Best regards, -- Ceki On 23/09/2019 17:39, John Morton wrote: > Hello again Logback users, > > I’m “bumping” this thread as I’m still having to use Logback 1.1.3 for > my application. I’d like a bug to be opened in Logback if anyone could > confirm whether what I am trying to do “should work”. I do not know if > the way I set it up in Logback 1.1.3 is “supported” or not. > > Please let me know, thank you so much! > > - John > > *From: *John Morton <jtmorton@calpoly.edu> > *Date: *Wednesday, December 19, 2018 at 7:14 PM > *To: *"logback-user@qos.ch" <logback-user@qos.ch> > *Subject: *Multiple FileAppenders writing to single file no longer > working in Logback 1.2.3 > > Hello there Logback users. An application I’m responsible uses logback > for its internal logging. It logs to various files at various levels. > I’ve extended it successfully to reformat certain log messages and save > them into a new log called “combined-audit.log”. This is all within a > single JVM, logging to a file on a local filesystem. > > The reason for the three loggers/appenders is that I want to log from > three different packages, at different levels, filter each differently, > and encode each differently. This is all necessary because I cannot make > changes to the app itself, so I’m trying to use Logback to accomplish my > needs. > > The below configuration works in Logback 1.1.3 and not in Logback 1.2.3. > > In Logback 1.2.3, only messages from one of the three loggers are logged > to the single file. When I change the file name to three different > files, messages from each logger are in each file. > > I read through release notes from 1.1.3 to 1.2.3 and nothing jumped out > at me as having the potential to change this behavior. > > Is this a supported configuration in 1.2.3? Does anyone have any ideas > for how to accomplish the objective? > > <!-- Appends app audit messages to combined audit log --> > > <appender name="COMBINED_AUDIT" class="ch.qos.logback.core.FileAppender"> > > <File>${logging.directory}/combined-audit.log</File> > > <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> > > <charset>UTF-8</charset> > > > <Pattern>%date{ISO8601}|%mdc{m.remote_addr}|%mdc{m.jsessionid}||%msg%n</Pattern> > > </encoder> > > </appender> > > <logger name="j.k.l" level="ALL" additivity="false"> > > <appender-ref ref="COMBINED_AUDIT"/> > > </logger> > > <!-- Appends LDAP auth messages to combined audit log --> > > <appender name="COMBINED_LDAP" class="ch.qos.logback.core.FileAppender"> > > <File>${logging.directory}/combined-audit.log</File> > > <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> > > <charset>UTF-8</charset> > > > <Pattern>%date{ISO8601}|%mdc{m.remote_addr}|%mdc{m.jsessionid}|%replace(%replace(%replace(%msg){'^.*resultCode=',''}){',.*identifier=','|'}){', > context=.*$',''}|LDAP|||%n</Pattern> > > </encoder> > > <filter class="ch.qos.logback.core.filter.EvaluatorFilter"> > > <evaluator> > > <expression>return message.contains("authenticate > response=");</expression> > > </evaluator> > > <OnMatch>ACCEPT</OnMatch> > > <OnMismatch>DENY</OnMismatch> > > </filter> > > </appender> > > <logger name="a.b.c.d" level="DEBUG" additivity="false"> > > <appender-ref ref="COMBINED_LDAP"/> > > </logger> > > <!-- Appends LDAP invalid username messages to combined audit log --> > > <appender name="COMBINED_INVALID_USERNAME" > class="ch.qos.logback.core.FileAppender"> > > <File>${logging.directory}/combined-audit.log</File> > > <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> > > <charset>UTF-8</charset> > > > <Pattern>%date{ISO8601}|%mdc{m.remote_addr}|%mdc{m.jsessionid}|INVALID_USERNAME||LDAP|||%n</Pattern> > > </encoder> > > <filter class="ch.qos.logback.core.filter.EvaluatorFilter"> > > <evaluator> > > <expression>return message.contains("failed using > filter=");</expression> > > </evaluator> > > <OnMatch>ACCEPT</OnMatch> > > <OnMismatch>DENY</OnMismatch> > > </filter> > > </appender> > > <logger name="v.w.x.y.z" level="INFO" additivity="false"> > > <appender-ref ref="COMBINED_INVALID_USERNAME"/> > > </logger> > > > _______________________________________________ logback-user mailing list logback-user@qos.ch http://mailman.qos.ch/mailman/listinfo/logback-user
Comments inline. On 24.09.2019 18:31, John Morton wrote:
Ceki,
I know how to have a single appender with a complex filter that would work to isolate the messages. The issue though is that I believe each appender can only have a single encoder. Due to the origin/content of the different types of messages, I'm using different encoders for each of the three types (in each of the three appenders). The only way I see being able to do this is with multiple appenders. Can you think of another way to do this with logback?
Are the encoders custom? Anyway, you could have an encoder which encodes in different ways depending on the event.
I've tried prudent mode on in 1.2.3 and it does not seem to make a difference. Was your 1.3 comment meant to indicate that the behavior should be the same as in 1.1.3 that I relied on previously?
No, 1.2 and 1.3 simply do not allow multiple appenders writing to the same file. -- Ceki Gülcü
participants (3)
-
Ceki -
Ceki Gülcü
-
John Morton