I've just finished reading that chapter. At first glance it seems that using a combination of JNDIBasedContextDiscriminator and SiftingAppender should solve my problem ? But then I read on through the example (Yoda, Kenobi) and now I'm confused. Especially because of the following sentence from the example "Note that even if the ch.qos.starwars.shared.Mustafar logger outputs to kenobi.log it is still attached to 'yoda'. Thus, we have two distinct logging contexts logging to the same file, in this case kenobi.log. Each of these contexts reference FileAppender instances, nested within distinct SiftingAppender instances, logging to the same file." So although the solution described (JNDIBasedContextDiscriminator and SiftingAppender) is about how to log to different files from within a shared codebase, the example doesn't seem to support that theory because it says that two distinct logging contexts are logging to the same file. Is the example wrong ? What am I missing ? Edwin -----Original Message----- From: logback-user-bounces@qos.ch [mailto:logback-user-bounces@qos.ch] On Behalf Of Ceki Gülcü Sent: woensdag 28 april 2010 17:28 To: logback users list Subject: Re: [logback-user] Log separation Hello Edwin, The chapter on logging separation [1] should be helpful, in particular the section entitled "Taming static references in shared libraries". HTH, [1] http://logback.qos.ch/manual/loggingSeparation.html On 28/04/2010 4:56 PM, Dhondt, Edwin wrote:

I've got 4 web applications. They all have some specific code containing log statements but they also include a common "backend" codebase jar. Code in that common codebase also contains log statements.

I want each of those 4 applications to have their own log file. That is, I want all log messages for webapp 1 to appear in webapp 1's log file, messages for webapp2 in webapp 2's file and so on.

This is not a problem for the log statements appearing in the code specific to each webapp.

But what about log statements in de common "backend" codebase (included by each of the webapps) ? How do those log statements "know" when log statements should go to webapp 1's logfile or webapp 2's log file ? Because that depends on where the calling thread initiated (in webapp1, 2, ...). I could augment each of my backend method signatures with a extra context parameter, but I hope there are more transparent solutions ?

User => methodA Webapp1 => backend codebase methodX => webapp1.log User => methodB Webapp2 => backend codebase methodX => webapp2.log ...

Any advice would be welcome. Thanks, EDH

_______________________________________________ Logback-user mailing list Logback-user@qos.ch http://qos.ch/mailman/listinfo/logback-user

Isn't the chapter about the situation where you have different web applications that share a common library and more specifically about how to enable logging to go to distinct log files ( a yoda and a Kenobi file) in that particular setting ? If so, then I don't understand why in Kenobi.log, entries with yoda context appear and why it's stated that logging occurs to the same file ? I would have expected that logging within a thread launched "at" yoda would result in log messages to go exclusively to yoda.log, and that logging within a thread launched "at" Kenobi would result in log messages going exclusively to Kenobi.log ? Can you shed a light on that ? What am I not understanding ? -----Original Message----- From: logback-user-bounces@qos.ch [mailto:logback-user-bounces@qos.ch] On Behalf Of Ceki Gülcü Sent: woensdag 28 april 2010 17:50 To: logback users list Subject: Re: [logback-user] Log separation The example is not wrong. You probably read the chapter too quickly. On 28/04/2010 5:41 PM, Dhondt, Edwin wrote:

I've just finished reading that chapter.

At first glance it seems that using a combination of JNDIBasedContextDiscriminator and SiftingAppender should solve my problem ?

But then I read on through the example (Yoda, Kenobi) and now I'm confused. Especially because of the following sentence from the example "Note that even if the ch.qos.starwars.shared.Mustafar logger outputs to kenobi.log it is still attached to 'yoda'. Thus, we have two distinct logging contexts logging to the same file, in this case kenobi.log. Each of these contexts reference FileAppender instances, nested within distinct SiftingAppender instances, logging to the same file." So although the solution described (JNDIBasedContextDiscriminator and SiftingAppender) is about how to log to different files from within a shared codebase, the example doesn't seem to support that theory because it says that two distinct logging contexts are logging to the same file. Is the example wrong ? What am I missing ?

Edwin

-----Original Message----- From: logback-user-bounces@qos.ch [mailto:logback-user-bounces@qos.ch] On Behalf Of Ceki Gülcü Sent: woensdag 28 april 2010 17:28 To: logback users list Subject: Re: [logback-user] Log separation

Hello Edwin,

The chapter on logging separation [1] should be helpful, in particular the section entitled "Taming static references in shared libraries".

HTH,

[1] http://logback.qos.ch/manual/loggingSeparation.html

On 28/04/2010 4:56 PM, Dhondt, Edwin wrote:

...

I've got 4 web applications. They all have some specific code containing log statements but they also include a common "backend" codebase jar. Code in that common codebase also contains log statements.

I want each of those 4 applications to have their own log file. That is, I want all log messages for webapp 1 to appear in webapp 1's log file, messages for webapp2 in webapp 2's file and so on.

This is not a problem for the log statements appearing in the code specific to each webapp.

But what about log statements in de common "backend" codebase (included by each of the webapps) ? How do those log statements "know" when log statements should go to webapp 1's logfile or webapp 2's log file ? Because that depends on where the calling thread initiated (in webapp1, 2, ...). I could augment each of my backend method signatures with a extra context parameter, but I hope there are more transparent solutions ?

User => methodA Webapp1 => backend codebase methodX => webapp1.log User => methodB Webapp2 => backend codebase methodX => webapp2.log ...

Any advice would be welcome. Thanks, EDH

_______________________________________________ Logback-user mailing list Logback-user@qos.ch http://qos.ch/mailman/listinfo/logback-user

_______________________________________________ Logback-user mailing list Logback-user@qos.ch http://qos.ch/mailman/listinfo/logback-user

_______________________________________________ Logback-user mailing list Logback-user@qos.ch http://qos.ch/mailman/listinfo/logback-user

That clarifies a lot, I hope ;-). My comprehension is that the intention of this all is to separate the logfiles in a scenario where you have multiple webapps "sharing" a common library (jar). In that scenario the intention is to have log messages produced by bóth webapp-specific code ánd log messages produced by log statements in the common library code to appear in distinct logfiles. E.g. log statements produced "in the context of" webapp1 (e.g. Yoda) must ALL (that is both log messages produced in yoda ánd log messages produced by code in the common codebase called from yoda) go to yoda.log. log statements produced "in the context of" webapp2 (e.g. Kenobi) must ALL (that is both log messages produced in kenobi ánd log messages produced by code in the common codebase called from kenobi) go to kenobi.log. E.g let's say webapp1 and webapp2 invoke common code to create a user (createUser). If a call createUser from "within" webapp1, I want the log messages produced in createUser, like "user x created" to go to the webapp1.log file, when I call createUser from "within" webapp2, I want the log messages to go to the webapp2.log file, and not for some reason or the other (e.g. webapp1 initalized before webapp2), to webapp1's log file. That is, coming back to the starwars example when I open the yoda.log I only want to see log messages produced in the context of running the Yoda webapp. That is, when I open the kenobi.log I only want to see log messages produced in the context of running the Kenobi webapp. Is the above interpretation correct ? If yes, then I still don't see how the ASCII diagram and the example in the logback manual (***) support the above: (***) " If kenobi and yoda are web-applications ... Visiting http://localhost:port/kenobi/servlet will output the following in kenobi.log. DEBUG ch.qos.starwars.kenobi.KenobiServlet cn=kenobi - in doGet() DEBUG ch.qos.starwars.shared.Mustafar cn=yoda - in foo() Note that even if the ch.qos.starwars.shared.Mustafar logger outputs to kenobi.log it is still attached to 'yoda'. Thus, we have two distinct logging contexts logging to the same file, in this case kenobi.log. "

From the diagram and the above excerpt (***) of the example in the manual I can only deduce that log messages produced "in the context of" Kenobi will be scattered over two files because when the Yoda app is initialized the first, the Mustafar logger will be "bound" to the Yoda context, not the Kenobi context. Thereby, log messages produced while running Kenobi will be scattered over yoda.log and Kenobi.log. Is that correct ? If yes, then there's no log separation.

Please clarify. Thanks, Edwin -----Original Message----- From: logback-user-bounces@qos.ch [mailto:logback-user-bounces@qos.ch] On Behalf Of Ceki Gülcü Sent: woensdag 28 april 2010 19:34 To: logback users list Subject: Re: [logback-user] Log separation Here is an ASCII diagram which might clarify the point. +---------------------+ +-------------------------+ | yoda context | | kenobi context | +---------------------+ +-------------------------+ | | | | +-------------------+ +------------+ +-----------+ +-----------+ |FILE-yoda appender | |FILE-Kenobi | | FILE-yoda | |FILE-kenobi| +-------------------+ +------------+ +-----------+ +-----------+ | | | | | | | | | | | | | v | | | kenobi.log (file) <-------|--------------- + v | yoda.log (file) <---------------------------+ Due to SiftingAppenders (one per context) there are two distinct FileAppenders (one in each context) writing to the same file. There are four FileAppenders (two per context). Do not cling to the term "same file" as there is a whole explanatory context. If you still do not see the point, that's probably because you are unfamiliar with SiftingAppender. HTH On 28/04/2010 6:14 PM, Dhondt, Edwin wrote:

Isn't the chapter about the situation where you have different web applications that share a common library and more specifically about how to enable logging to go to distinct log files ( a yoda and a Kenobi file) in that particular setting ? If so, then I don't understand why in Kenobi.log, entries with yoda context appear and why it's stated that logging occurs to the same file ? I would have expected that logging within a thread launched "at" yoda would result in log messages to go exclusively to yoda.log, and that logging within a thread launched "at" Kenobi would result in log messages going exclusively to Kenobi.log ? Can you shed a light on that ? What am I not understanding ?

-----Original Message----- From: logback-user-bounces@qos.ch [mailto:logback-user-bounces@qos.ch] On Behalf Of Ceki Gülcü Sent: woensdag 28 april 2010 17:50 To: logback users list Subject: Re: [logback-user] Log separation

The example is not wrong. You probably read the chapter too quickly.

On 28/04/2010 5:41 PM, Dhondt, Edwin wrote:

...

I've just finished reading that chapter.

At first glance it seems that using a combination of JNDIBasedContextDiscriminator and SiftingAppender should solve my problem ?

But then I read on through the example (Yoda, Kenobi) and now I'm confused. Especially because of the following sentence from the example "Note that even if the ch.qos.starwars.shared.Mustafar logger outputs to kenobi.log it is still attached to 'yoda'. Thus, we have two distinct logging contexts logging to the same file, in this case kenobi.log. Each of these contexts reference FileAppender instances, nested within distinct SiftingAppender instances, logging to the same file." So although the solution described (JNDIBasedContextDiscriminator and SiftingAppender) is about how to log to different files from within a shared codebase, the example doesn't seem to support that theory because it says that two distinct logging contexts are logging to the same file. Is the example wrong ? What am I missing ?

Edwin

-----Original Message----- From: logback-user-bounces@qos.ch [mailto:logback-user-bounces@qos.ch] On Behalf Of Ceki Gülcü Sent: woensdag 28 april 2010 17:28 To: logback users list Subject: Re: [logback-user] Log separation

Hello Edwin,

The chapter on logging separation [1] should be helpful, in particular the section entitled "Taming static references in shared libraries".

HTH,

[1] http://logback.qos.ch/manual/loggingSeparation.html

On 28/04/2010 4:56 PM, Dhondt, Edwin wrote:

...

I've got 4 web applications. They all have some specific code containing log statements but they also include a common "backend" codebase jar. Code in that common codebase also contains log statements.

I want each of those 4 applications to have their own log file. That is, I want all log messages for webapp 1 to appear in webapp 1's log file, messages for webapp2 in webapp 2's file and so on.

This is not a problem for the log statements appearing in the code specific to each webapp.

But what about log statements in de common "backend" codebase (included by each of the webapps) ? How do those log statements "know" when log statements should go to webapp 1's logfile or webapp 2's log file ? Because that depends on where the calling thread initiated (in webapp1, 2, ...). I could augment each of my backend method signatures with a extra context parameter, but I hope there are more transparent solutions ?

User => methodA Webapp1 => backend codebase methodX => webapp1.log User => methodB Webapp2 => backend codebase methodX => webapp2.log ...

Any advice would be welcome. Thanks, EDH

_______________________________________________ Logback-user mailing list Logback-user@qos.ch http://qos.ch/mailman/listinfo/logback-user

_______________________________________________ Logback-user mailing list Logback-user@qos.ch http://qos.ch/mailman/listinfo/logback-user

_______________________________________________ Logback-user mailing list Logback-user@qos.ch http://qos.ch/mailman/listinfo/logback-user _______________________________________________ Logback-user mailing list Logback-user@qos.ch http://qos.ch/mailman/listinfo/logback-user

_______________________________________________ Logback-user mailing list Logback-user@qos.ch http://qos.ch/mailman/listinfo/logback-user

On 29/04/2010 9:33 AM, Ralph Goers wrote:

What you are asking to do can be done by having a logger for the classes in the common jar (presumably they are all using the same package name at some level) that is configured with 2 appenders. Each appender is then configured with a filter so that it only allows data from the relevant webapp to be written to that particular file. If you are capturing the request URL or something else that identifies the particular webapp and placing it in the MDC then you can use the EvaluatorFilter (or write your own) to do the filtering.

That's essentially what SiftingAppender does. Note that your solution suffers from the same exact contention problem that Edwin seems to dislike. Also in [1] you only need to configure one SiftingAppender per web-app and it will transparently deal with the order in which web-apps are started. In your approach, either you have to elect one web-app, say ADAM, to start first and in ADAMS's logback.xml configure one additional appender per web-app. Whenever a webb-app is added you would need to adapt ADAM's web-app by hand. If you can't elect a web-app to start first, then you would need to add an appender per web-app in each web-app by hand which would be horrible to maintain beyond two web-apps. The solution described in [1] is not so different than yours, except that it automates a lot of the grunt work. [1] http://logback.qos.ch/manual/loggingSeparation.html -- Ceki

On 29/04/2010 10:43 AM, Ralph Goers wrote:

Sorry, I've never used the SiftingAppender. Just read the doc for it. Yes, they are similar and it is a better approach. Although the doc mentions using it for static loggers it occurs to me that it is probably also useful if all the web apps are configured to use the same config file even if all the jars are in the web app.

If all the jars are in the same web-app (shared jars), then you don't need ContextJNDISelector nor SiftingAppender for secondary separation. If you wish to share the same logback.xml file for all web-apps, I'd use a prperty to set the context name. For example, <configuration> <contextName>${CONTEXT_NAME}</contextName> <appender name="FILE" class="ch.qos.logback.core.FileAppender"> <file>${CONTEXT_NAME}.log</file> <encoder> <pattern>%d [%thread] %-5level %logger{35} - %msg%n</pattern> </encoder> </appender> </configuration> The question is then how do you set the CONTEXT_NAME property. You could do so before invoking joran configurator, typically in a ServletContextListener. Here is sample code: public class MyServletContextListener implements ServletContextListener { public void contextInitialized(ServletContextEvent sce) { LoggerContext lc = (LoggerContext) LoggerFactory.getILoggerFactory(); JoranConfigurator configurator = new JoranConfigurator(); configurator.setContext(lc); // undo default configuration lc.reset(); lc.putProperty("CONTEXT_NAME", the_name_of_this_context); configurator.doConfigure(args[0]); } } If you only want to rely on automatic configuration, then defining properties on the fly could be adequate [1]. Your config file becomes: <configuration> <define name="CONTEXT_NAME" class="class.implementing.PropertyDefiner.ReturningAContextName"> <contextName>${CONTEXT_NAME}</contextName> <appender name="FILE" class="ch.qos.logback.core.FileAppender"> <file>${CONTEXT_NAME}.log</file> <encoder> <pattern>%d [%thread] %-5level %logger{35} - %msg%n</pattern> </encoder> </appender> </configuration> The "class.implementing.PropertyDefiner.ReturningAContextName" would need to figure out the name of the enclosing web-app via an adhoc mechanism, e.g. via a resource look up containing the name of the web-app. -- Ceki

So can I be sure that when following all the steps in [1] the yoda web-app will only write to yoda.log and the Kenobi web-app will only write to Kenobi.log indepent on which application is started first and calls the shared code the first ? That is, can I be sure that when following [1] I won't have any contention because yoda threads will write to yoda.log and Kenobi-initiated threads will write to Kenobi.log ? Edwin -----Original Message----- From: logback-user-bounces@qos.ch [mailto:logback-user-bounces@qos.ch] On Behalf Of Ceki Gülcü Sent: donderdag 29 april 2010 10:44 To: logback users list Subject: Re: [logback-user] Log separation On 29/04/2010 8:59 AM, Dhondt, Edwin wrote: [snip]

That is, coming back to the starwars example when I open the yoda.log I only want to see log messages produced in the context of running the Yoda webapp. That is, when I open the kenobi.log I only want to see log messages produced in the context of running the Kenobi webapp.

Is the above interpretation correct ?

Yes, that is the problem solved by [1]. [snip]

From the diagram and the above excerpt (***) of the example in the manual I can only deduce that log messages produced "in the context of" Kenobi will be scattered over two files because when the Yoda app is initialized the first, the Mustafar logger will be "bound" to the Yoda context, not the Kenobi context. Thereby, log messages produced while running Kenobi will be scattered over yoda.log and Kenobi.log. Is that correct ? If yes, then there's no log separation.

There is log separation because of SiftingAppender, that's the whole beauty of the approach. Separation is done at two levels. First at logger retrieval time by ContextJNDISelector which will return loggers attaching them to the context of the web-app. This works well for instance loggers in non-shared classes. For static loggers in shared classes which are bound to the first context that calls them, SiftingAppender performs a second level of separation. HTH, [1] http://logback.qos.ch/manual/loggingSeparation.html -- Ceki _______________________________________________ Logback-user mailing list Logback-user@qos.ch http://qos.ch/mailman/listinfo/logback-user

Given [1] http://logback.qos.ch/manual/loggingSeparation.html and especially the part BEFORE the sentence "So are we done yet? Can we declare victory and go home? Well, not quite." I would say yes. Given the the part AFTER the sentence "So are we done yet? Can we declare victory and go home? Well, not quite." I would say no. Because in that paragraph one starts talking about: [snip] "Note that even if the ch.qos.starwars.shared.Mustafar logger outputs to kenobi.log it is still attached to 'yoda'. Thus, we have two distinct logging contexts logging to the same file, in this case kenobi.log. Each of these contexts reference FileAppender instances, nested within distinct SiftingAppender instances, logging to the same file. Although logging separation seems to function according to our wishes, FileAppender instances cannot safely write to the same file unless they enable prudent mode. Otherwise, the target file will be corrupted." [/snip] That is one starts talking about the fact different logging contexts are logging to the same file, and that file appenders are writing to the same file and that the latter is not save and that therefore prudent mode should be used. Too me, the latter indicates there is no log separation anymore in the scenario described. Is that correct ? If not, where am I missing the point ? -----Original Message----- From: logback-user-bounces@qos.ch [mailto:logback-user-bounces@qos.ch] On Behalf Of Ceki Gülcü Sent: donderdag 29 april 2010 11:39 To: logback users list Subject: Re: [logback-user] Log separation On 29/04/2010 10:55 AM, Dhondt, Edwin wrote:

So can I be sure that when following all the steps in [1] the yoda web-app will only write to yoda.log and the Kenobi web-app will only write to Kenobi.log indepent on which application is started first and calls the shared code the first ? That is, can I be sure that when following [1] I won't have any contention because yoda threads will write to yoda.log and Kenobi-initiated threads will write to Kenobi.log ?

Given the previous lengthy discussion, what do you think? -- Ceki _______________________________________________ Logback-user mailing list Logback-user@qos.ch http://qos.ch/mailman/listinfo/logback-user

Tools like Splunk are indeed an alternative. However if I have two apps deployed using a shared library (jar) that also contains log statements, I want those log statements to appear in webapp1's or webapp2's log files, depending on whether the common code was called in the context of webapp1 or 2. I want each of the webapps writing to a different file. That is, I don't want n applications writing to the same file. -----Original Message----- From: logback-user-bounces@qos.ch [mailto:logback-user-bounces@qos.ch] On Behalf Of Ralph Goers Sent: donderdag 29 april 2010 2:15 To: logback users list Subject: Re: [logback-user] Log separation Importance: High I find myself wondering if you really want what you say you do. If you want to be able to view all the activity for a particular webapp a very practical way to do this is to use a tool like Splunk that can aggregate all the log data and then report based on filtering the data. Typically these tools are designed to take logs from various systems such as app servers, web servers, applications, etc. and correlate the data based on things like common a common session id value or other identifiers you put into your log records. So instead of doing the segregation based on filters when the logs are written you do the segregation at the time you are viewing the logs. The advantage of this is that while what you are asking to do may work for a small number of servers it becomes terribly hard to do as you scale up. In addition, in my experience if you try to write to a single log file from 2 applications running on different servers you are likely to experience problems over time. Ralph On Apr 28, 2010, at 7:56 AM, Dhondt, Edwin wrote:

I've got 4 web applications. They all have some specific code containing log statements but they also include a common "backend" codebase jar. Code in that common codebase also contains log statements.

I want each of those 4 applications to have their own log file. That is, I want all log messages for webapp 1 to appear in webapp 1's log file, messages for webapp2 in webapp 2's file and so on.

This is not a problem for the log statements appearing in the code specific to each webapp.

But what about log statements in de common "backend" codebase (included by each of the webapps) ? How do those log statements "know" when log statements should go to webapp 1's logfile or webapp 2's log file ? Because that depends on where the calling thread initiated (in webapp1, 2, ...). I could augment each of my backend method signatures with a extra context parameter, but I hope there are more transparent solutions ?

User => methodA Webapp1 => backend codebase methodX => webapp1.log User => methodB Webapp2 => backend codebase method => webapp2.log ...

Any advice would be welcome. Thanks, EDH

_______________________________________________ Logback-user mailing list Logback-user@qos.ch http://qos.ch/mailman/listinfo/logback-user

_______________________________________________ Logback-user mailing list Logback-user@qos.ch http://qos.ch/mailman/listinfo/logback-user