- logback-dev - mailman3.qos.ch

Improving Joran's RuleStore performance
by Niall Gallagher 20 Jun '13

20 Jun '13

Hi Ceki, Just wondering about the SUB-STRING patterns, such as "*/if/then/*" and "*/if/else/*". I'm currently working on adding a permuterm tree to the concurrent-trees[1] project. It will support fast retrieval from the tree for wildcard queries like: X, X*, *X, *X*, X*Y. These patterns would be fully accelerated. It looks like the *X* variant might fit your sub-string patterns. It's also possible to have partial acceleration for additional patterns such as X*Y*Z, *X*Y*Z, *X*Y*Z* and so on with any number of terms, which involves using the first and last terms to find a candidate set in the tree and then doing on-the-fly filtering for the terms in the middle. Would that help? I'm about 80% of the way to completion. [1] https://code.google.com/p/concurrent-trees/ Niall

2 2

Improving Joran's RuleStore performance
by ceki 20 Jun '13

20 Jun '13

Hello all, I am currently working on improving the performance of Joran rule store. As described in [1], Joran interprets XML files by applying rules while traversing the XML tree. A rule consists of a pattern and an action. While traversing the tree of XML elements, Joran asks its RuleStore for the Action applicable for the current element. Let assume that the RuleStore consists of just two rules rule1: "configuration" -> ConfigurationAction rule2: "configuration/statusListener" -> StatusListenerAction here is a short config file: <configuration> <statusListener class="..." /> <logger .../> </configuration> for the <configuration> element, the rule 1 matches and ConfigurationAction is applicable; for the <statusListener> element nested within <configuration> rule 2 matches and StatusListenerAction is applicable; for the <logger> element there is no matching rule. Joran's RuleStore understands 3 types of patterns: 1) EXACT patterns, such as "configuration" and "configuration/statusListener" 2) PREFIX patterns, such as configuration/appender/sift/* 3) SUFFIX patterns, such as "*/param", "*/if", "*if/then" 4) SUB-STRING patterns, such as "*/if/then/*" and "*/if/else/*" Currently there are 27 rules using exact patterns, one rule using a prefix pattern, four rules using suffix patterns and two rules using sub-string patterns. The rules are defined in [2] and [3]. The current implementation of the RuleStore [4] is rather naive. It iterates over the various patterns to see if there is a match. Placing the various patterns as keys in a trie seems like a good way of improving performance. BTW, Joran instances are not shared so that support for concurrent access is not required. Given that there are only a few rules, I was initially thinking of using trie nodes with 256 links. Assuming an average key length of 20 and 30 rules, that would use about 600x256 node pointers bytes or 1.2MB of memory (assuming 8byte pointers). However, since many of the patterns share the same prefixes, it makes sense to compress the trie, as done in the Patricia trie. Anyway, for exact and prefix patterns, the trie data structure seems highly appropriate. For suffix patterns, a reverse trie could do the job. For sub-strings patterns, I guess looping over the patterns would be acceptable (there are only two sub-string patterns). I should note that when the traversing the XML tree, often enough none of the rules are applicable and Joran falls back on implicit rules. Thus, in practice, much of the configuration work is done by implicit actions which are triggered when the search for an explicit rule fails. Enough said. Your comments, questions and ideas for improvement are most welcome, [1] http://logback.qos.ch/manual/onJoran.html [2] http://tinyurl.com/JoranConfiguratorBase [3] http://tinyurl.com/JoranConfigurator [4] http://tinyurl.com/SimpleRuleStore -- Ceki 65% of statistics are made up on the spot

2 1

[qos-ch/logback] 64c4e7: updated links
by Ceki Gulcu 20 Jun '13

20 Jun '13

Branch: refs/heads/master Home: https://github.com/qos-ch/logback Commit: 64c4e7d53b6fcf0578062b0c2974b62f0dfb9fbb https://github.com/qos-ch/logback/commit/64c4e7d53b6fcf0578062b0c2974b62f0d… Author: Ceki Gulcu <ceki(a)qos.ch> Date: 2013-06-20 (Thu, 20 Jun 2013) Changed paths: M logback-site/src/site/pages/index.html Log Message: ----------- updated links

1 0

[JIRA] (LOGBACK-868) Stuck Thread on logger method in Weblogic Servers
by Mahesh Chimanpure (JIRA) 20 Jun '13

20 Jun '13

1 0

[JIRA] (LOGBACK-871) Unable to load logback.xml in multiple classloader jvm.
by David Roussel (JIRA) 19 Jun '13

19 Jun '13

1 0

[JIRA] (LOGBACK-109) Locking issues when prudent logging to a (windows) clustered storage device on owner-switch
by Mahesh Chimanpure (JIRA) 19 Jun '13

19 Jun '13

1 0

[JIRA] (LOGBACK-871) Unable to load logback.xml in multiple classloader jvm.
by ThisadeeP (JIRA) 19 Jun '13

19 Jun '13

1 0

[JIRA] (LOGBACK-864) OverlappingFileLockException with Rolling file appender using prudent flag
by David Roussel (JIRA) 18 Jun '13

18 Jun '13

1 0

[JIRA] (LOGBACK-864) OverlappingFileLockException with Rolling file appender using prudent flag
by David Roussel (JIRA) 18 Jun '13

18 Jun '13

1 0

[JIRA] (LOGBACK-721) SizeAndTimeBasedFNATP doesn't set timestamp in logged files after rollover triggered by maximum size
by Alexey Pomelov (JIRA) 17 Jun '13

17 Jun '13

1 0