[JIRA] Created: (LBCLASSIC-53) Add MDC Level Filtering
Add MDC Level Filtering ----------------------- Key: LBCLASSIC-53 URL: http://jira.qos.ch/browse/LBCLASSIC-53 Project: logback-classic Issue Type: New Feature Environment: All Reporter: Ralph Goers Assignee: Logback dev list Attachments: MDCLevelFilter.zip The company I work for is an Application Service Provider. Requests to the same software running in a single JVM might be on behalf of any one of these companies. In addition, our middle tier contains services that my be called from several different products and are run on behalf of a user of one of these companies. Operationally, we require the ability to be able to filter based on company, product, possibly the end user or several other criteria. The TurboFilter provided allows individual values for keys in the MDC to be associated with a logging level. So if the MDC contains a key for the company the TurboFilter can be configured with the name of the company and the corresponding logging level. So if CompanyA's level is set to debug while the default level is error, only debug events for CompanyA will be passed on. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[ http://jira.qos.ch/browse/LBCLASSIC-53?page=com.atlassian.jira.plugin.system... ] Ralph Goers updated LBCLASSIC-53: --------------------------------- Attachment: MDCLevelFilter.zip TurboFilter that implements this feature.
Add MDC Level Filtering -----------------------
Key: LBCLASSIC-53 URL: http://jira.qos.ch/browse/LBCLASSIC-53 Project: logback-classic Issue Type: New Feature Environment: All Reporter: Ralph Goers Assignee: Logback dev list Attachments: MDCLevelFilter.zip
The company I work for is an Application Service Provider. Requests to the same software running in a single JVM might be on behalf of any one of these companies. In addition, our middle tier contains services that my be called from several different products and are run on behalf of a user of one of these companies. Operationally, we require the ability to be able to filter based on company, product, possibly the end user or several other criteria. The TurboFilter provided allows individual values for keys in the MDC to be associated with a logging level. So if the MDC contains a key for the company the TurboFilter can be configured with the name of the company and the corresponding logging level. So if CompanyA's level is set to debug while the default level is error, only debug events for CompanyA will be passed on.
-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Ralph, As you can see from my recent commits, I have started working on /LBCLASSIC-53. It makes use of Joran's capabilities so as to shorten and improve on the configuration syntax required the filer. It is a mere refactorisation of your work, without any change in behavior. However, I am wonder if behavioral changes are not called for. I can think of two major use cases. The logging system is mostly silent but we would like to increase verbosity for users Alice and Bob and keep it as is for the other users. The logging system is mostly verbose but we would like to decrease verbosity for certain requests, say typeA and typeB, and keep it as is for the other types of requests. Here I assume that the type of request is can be easily and accurately set on the MDC. Now, DynamicThresholdFilter, as a threshold, can easily accommodate case 2. Set the threshold to a high value for typeA and TypeB, and set the root level to a low level such as DEBUG or INFO. You are done. The second case cannot be accommodated with the current implementation of DynamicThresholdFilter because it returns NEUTRAL when the request level is higher or equal to the threshold level (computed dynamically from MDC). We would want to return ACCEPT instead. Do you see why? I also note that the defaultThresold value is not useful in any of the above use cases. Would you agree thus far? Ralph Goers (JIRA) wrote:
[ http://jira.qos.ch/browse/LBCLASSIC-53?page=com.atlassian.jira.plugin.system... ]
Ralph Goers updated LBCLASSIC-53: ---------------------------------
Attachment: MDCLevelFilter.zip
TurboFilter that implements this feature.
Add MDC Level Filtering -----------------------
Key: LBCLASSIC-53 URL: http://jira.qos.ch/browse/LBCLASSIC-53 Project: logback-classic Issue Type: New Feature Environment: All Reporter: Ralph Goers Assignee: Logback dev list Attachments: MDCLevelFilter.zip
The company I work for is an Application Service Provider. Requests to the same software running in a single JVM might be on behalf of any one of these companies. In addition, our middle tier contains services that my be called from several different products and are run on behalf of a user of one of these companies. Operationally, we require the ability to be able to filter based on company, product, possibly the end user or several other criteria. The TurboFilter provided allows individual values for keys in the MDC to be associated with a logging level. So if the MDC contains a key for the company the TurboFilter can be configured with the name of the company and the corresponding logging level. So if CompanyA's level is set to debug while the default level is error, only debug events for CompanyA will be passed on.
-- Ceki Gülcü QOS.ch is looking to hire talented developers located in Switzerland to work on cutting-edge software projects. If you think you are qualified, then please contact ceki@qos.ch.
Just to be clear. Ceki Gulcu wrote:
I can think of two major use cases.
This is the first:
The logging system is mostly silent but we would like to increase verbosity for users Alice and Bob and keep it as is for the other users.
and this is the second:
The logging system is mostly verbose but we would like to decrease verbosity for certain requests, say typeA and typeB, and keep it as is for the other types of requests. Here I assume that the type of request is can be easily and accurately set on the MDC. -- Ceki Gülcü
QOS.ch is looking to hire talented developers located in Switzerland to work on cutting-edge software projects. If you think you are qualified, then please contact ceki@qos.ch.
First, I hate using "high" and "low" for logging levels. I find it confusing as I usually think of Debug as high since it is a greater verbosity, rather than a lower numeric value. Next your description doesn't seem to describe how the filter actually works. I'm somewhat confused as well because in one paragraph you said "can easily accommodate case 2" and then in the next paragraph "the second case cannot be accommodated". I have to look at the code again but I'm pretty sure it should work differently than what you describe. It should accomodate both use case 1 and 2. Assume that the default level is set to Warn. Messages destined for user carol would be rejected by this filter (i.e. return DENY) unless they are Warn or Error messages. On the other hand, if users Alice and Bob are set to Debug then messages that match them will pass through the filter unless they are trace messages. However, we don't want to accept these just yet as the intention is not to log everything for these users, only to allow the rest of normal filtering to occur. To do that the filter must return NEUTRAL. This allows use case 1 to work correctly. Now, if the default level was Trace all messages would pass through the filter. However, if Alice and Bob are now set to Error messages related to them would not pass through the filter unless they were only Errors. This should satisfy use case 2. To The default level was not useful in any of the use cases you named because you didn't include any messages from Carol. In my usage of the filter I expect the default level to normally be set to Error. If the code doesn't work the way I described it needs to be corrected so it does. Ralph Ceki Gulcu wrote:
Ralph,
As you can see from my recent commits, I have started working on /LBCLASSIC-53. It makes use of Joran's capabilities so as to shorten and improve on the configuration syntax required the filer. It is a mere refactorisation of your work, without any change in behavior.
However, I am wonder if behavioral changes are not called for.
I can think of two major use cases.
The logging system is mostly silent but we would like to increase verbosity for users Alice and Bob and keep it as is for the other users.
The logging system is mostly verbose but we would like to decrease verbosity for certain requests, say typeA and typeB, and keep it as is for the other types of requests. Here I assume that the type of request is can be easily and accurately set on the MDC.
Now, DynamicThresholdFilter, as a threshold, can easily accommodate case 2. Set the threshold to a high value for typeA and TypeB, and set the root level to a low level such as DEBUG or INFO. You are done.
The second case cannot be accommodated with the current implementation of DynamicThresholdFilter because it returns NEUTRAL when the request level is higher or equal to the threshold level (computed dynamically from MDC). We would want to return ACCEPT instead. Do you see why?
I also note that the defaultThresold value is not useful in any of the above use cases.
Would you agree thus far?
Ralph Goers (JIRA) wrote:
[ http://jira.qos.ch/browse/LBCLASSIC-53?page=com.atlassian.jira.plugin.system... ]
Ralph Goers updated LBCLASSIC-53: ---------------------------------
Attachment: MDCLevelFilter.zip
TurboFilter that implements this feature.
Add MDC Level Filtering -----------------------
Key: LBCLASSIC-53 URL: http://jira.qos.ch/browse/LBCLASSIC-53 Project: logback-classic Issue Type: New Feature Environment: All Reporter: Ralph Goers Assignee: Logback dev list Attachments: MDCLevelFilter.zip
The company I work for is an Application Service Provider. Requests to the same software running in a single JVM might be on behalf of any one of these companies. In addition, our middle tier contains services that my be called from several different products and are run on behalf of a user of one of these companies. Operationally, we require the ability to be able to filter based on company, product, possibly the end user or several other criteria. The TurboFilter provided allows individual values for keys in the MDC to be associated with a logging level. So if the MDC contains a key for the company the TurboFilter can be configured with the name of the company and the corresponding logging level. So if CompanyA's level is set to debug while the default level is error, only debug events for CompanyA will be passed on.
Ralph Goers wrote: [snip]
Next your description doesn't seem to describe how the filter actually works. I'm somewhat confused as well because in one paragraph you said "can easily accommodate case 2" and then in the next paragraph "the second case cannot be accommodated".
Ooops: What I meant to say was: I can think of two major use cases. Case 1) The logging system is mostly silent but we would like to increase verbosity for users Alice and Bob and keep it as is for the other users. Case 2) The logging system is mostly verbose but we would like to decrease verbosity for certain requests, say typeA and typeB, and keep it as is for the other types of requests. Here I assume that the type of request is can be easily and accurately set on the MDC. Now, DynamicThresholdFilter, as a threshold, can easily accommodate case 2. Set the threshold high (e.g. ERROR) for typeA and TypeB, and set the root level to a low level such as DEBUG or INFO. You are done. The *first* case cannot be accommodated with the current implementation of DynamicThresholdFilter because it returns NEUTRAL when the request level is higher or equal to the threshold level (computed dynamically from MDC). We would want to return ACCEPT instead. Do you see why? I also note that the defaultThresold value is not useful in any of the above use cases.
I have to look at the code again but I'm pretty sure it should work differently than what you describe. It should accomodate both use case 1 and 2.
Assume that the default level is set to Warn. Messages destined for user carol would be rejected by this filter (i.e. return DENY) unless they are Warn or Error messages. On the other hand, if users Alice and Bob are set to Debug then messages that match them will pass through the filter unless they are trace messages. However, we don't want to accept these just yet as the intention is not to log everything for these users, only to allow the rest of normal filtering to occur. To do that the filter must return NEUTRAL. This allows use case 1 to work correctly.
I see. So in this case the default level is quite crucial. Your point about letting normal processing to occur also makes sense. However, it must be assumed that normal processing lets messages through, if root level set to error, having DynamicThresholdFilter return NEURAL does not help. Of course, if normal processing lets messages go through, then the current impl of DynamicThresholdFilter fits the bill nicely.
Now, if the default level was Trace all messages would pass through the filter. However, if Alice and Bob are now set to Error messages related to them would not pass through the filter unless they were only Errors. This should satisfy use case 2.
Yes, indeed. In LevelFilter [1], there is a possibility of returning ACCEPT, DENY or NEUTRAL either onMatch or onMisMatch. This allows for 6 onMatch/onMismatch combinations. In the current impl, we only use 1 out of the 6. If common use cases can be covered by just 1 combination, that's fine with me. :-) [1] http://logback.qos.ch/apidocs/ch/qos/logback/classic/filter/LevelFilter.html -- Ceki Gülcü QOS.ch is looking to hire talented developers located in Switzerland to work on cutting-edge software projects. If you think you are qualified, then please contact ceki@qos.ch.
While testing I ran into a problem with TimeBasedRollingTest. It got an EOFException. I couldn't reproduce it by running the test individually, but while doing that I noticed that the test is creating files in src/test/output. Why is it doing that? That is very bad form, especially since the files are not deleted by doing a mvn clean. These should be somewhere under the target directory. Ralph
Hello Ralph, Point well taken. Ralph Goers wrote:
While testing I ran into a problem with TimeBasedRollingTest. It got an EOFException. I couldn't reproduce it by running the test individually, but while doing that I noticed that the test is creating files in src/test/output. Why is it doing that? That is very bad form, especially since the files are not deleted by doing a mvn clean. These should be somewhere under the target directory.
Ralph
-- Ceki Gülcü QOS.ch is looking to hire talented developers located in Switzerland to work on cutting-edge software projects. If you think you are qualified, then please contact ceki@qos.ch.
Fixed in 1726. Ceki Gulcu wrote:
Hello Ralph,
Point well taken.
Ralph Goers wrote:
While testing I ran into a problem with TimeBasedRollingTest. It got an EOFException. I couldn't reproduce it by running the test individually, but while doing that I noticed that the test is creating files in src/test/output. Why is it doing that? That is very bad form, especially since the files are not deleted by doing a mvn clean. These should be somewhere under the target directory.
Ralph
-- Ceki Gülcü QOS.ch is looking to hire talented developers located in Switzerland to work on cutting-edge software projects. If you think you are qualified, then please contact ceki@qos.ch.
[ http://jira.qos.ch/browse/LBCLASSIC-53?page=com.atlassian.jira.plugin.system... ] Ralph Goers updated LBCLASSIC-53: --------------------------------- Attachment: dynamic.patch Patch DynamicThresholdFilter to allow onMatch and onMismatch to be configured.
Add MDC Level Filtering -----------------------
Key: LBCLASSIC-53 URL: http://jira.qos.ch/browse/LBCLASSIC-53 Project: logback-classic Issue Type: New Feature Environment: All Reporter: Ralph Goers Assignee: Logback dev list Attachments: dynamic.patch, MDCLevelFilter.zip
The company I work for is an Application Service Provider. Requests to the same software running in a single JVM might be on behalf of any one of these companies. In addition, our middle tier contains services that my be called from several different products and are run on behalf of a user of one of these companies. Operationally, we require the ability to be able to filter based on company, product, possibly the end user or several other criteria. The TurboFilter provided allows individual values for keys in the MDC to be associated with a logging level. So if the MDC contains a key for the company the TurboFilter can be configured with the name of the company and the corresponding logging level. So if CompanyA's level is set to debug while the default level is error, only debug events for CompanyA will be passed on.
-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[ http://jira.qos.ch/browse/LBCLASSIC-53?page=com.atlassian.jira.plugin.system... ] Ceki Gulcu resolved LBCLASSIC-53. --------------------------------- Fix Version/s: 0.9.10 Resolution: Fixed Patches reviewed and applied.
Add MDC Level Filtering -----------------------
Key: LBCLASSIC-53 URL: http://jira.qos.ch/browse/LBCLASSIC-53 Project: logback-classic Issue Type: New Feature Environment: All Reporter: Ralph Goers Assignee: Logback dev list Fix For: 0.9.10
Attachments: dynamic.patch, MDCLevelFilter.zip
The company I work for is an Application Service Provider. Requests to the same software running in a single JVM might be on behalf of any one of these companies. In addition, our middle tier contains services that my be called from several different products and are run on behalf of a user of one of these companies. Operationally, we require the ability to be able to filter based on company, product, possibly the end user or several other criteria. The TurboFilter provided allows individual values for keys in the MDC to be associated with a logging level. So if the MDC contains a key for the company the TurboFilter can be configured with the name of the company and the corresponding logging level. So if CompanyA's level is set to debug while the default level is error, only debug events for CompanyA will be passed on.
-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
participants (4)
-
Ceki Gulcu -
Ceki Gulcu (JIRA) -
Ralph Goers -
Ralph Goers (JIRA)