[JIRA] Created: (LBCLASSIC-198) DBAppender: Support long event_id values in logging_event

DBAppender: Support long event_id values in logging_event --------------------------------------------------------- Key: LBCLASSIC-198 URL: http://jira.qos.ch/browse/LBCLASSIC-198 Project: logback-classic Issue Type: Improvement Components: appender Affects Versions: 0.9.18 Reporter: Michael Lynch Assignee: Logback dev list Want to retain longer history of logging events. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCLASSIC-198?page=com.atlassian.jira.plugin.syste... ] Michael Lynch updated LBCLASSIC-198: ------------------------------------ Attachment: DBAppenderBase.java DBAppender.java Attached changes to core.db.DBAppenderBase and classic.db.DBAppender.
DBAppender: Support long event_id values in logging_event ---------------------------------------------------------
Key: LBCLASSIC-198 URL: http://jira.qos.ch/browse/LBCLASSIC-198 Project: logback-classic Issue Type: Improvement Components: appender Affects Versions: 0.9.18 Reporter: Michael Lynch Assignee: Logback dev list Attachments: DBAppender.java, DBAppenderBase.java
Want to retain longer history of logging events.
-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCLASSIC-198?page=com.atlassian.jira.plugin.syste... ] Ceki Gulcu commented on LBCLASSIC-198: -------------------------------------- In java 'int' is 32 bits, ranging from approx -2Billion to 2billion. Why would you ever want 'long' (64bit) event_id values?
DBAppender: Support long event_id values in logging_event ---------------------------------------------------------
Key: LBCLASSIC-198 URL: http://jira.qos.ch/browse/LBCLASSIC-198 Project: logback-classic Issue Type: Improvement Components: appender Affects Versions: 0.9.18 Reporter: Michael Lynch Assignee: Logback dev list Attachments: DBAppender.java, DBAppenderBase.java
Want to retain longer history of logging events.
-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

It is mostly out of caution with some amount of fatalism mixed in. We are working with a distributed databases that is synchronized back to a central database. In order to avoid key collisions we are partitioning the key space. We may have as many as 100 remote databases - reducing the key space for any one to ~21million. While I don't expect 21 million errors from a single instance (nor even all together), there are processes that can run every few seconds that can generate an error. Left unchecked/unnoticed for long enough and the volume of errors could add up over time. In my experience there is very little memory or disk space issues created by using a long instead of an int - although I recognize that logback is used in many different ways/environments, so my experience may not be as accurate in the context of logback. - Michael On Mon, Mar 22, 2010 at 12:52 PM, Ceki Gulcu (JIRA) <noreply-jira@qos.ch> wrote:
[ http://jira.qos.ch/browse/LBCLASSIC-198?page=com.atlassian.jira.plugin.syste... ]
Ceki Gulcu commented on LBCLASSIC-198: --------------------------------------
In java 'int' is 32 bits, ranging from approx -2Billion to 2billion. Why would you ever want 'long' (64bit) event_id values?
DBAppender: Support long event_id values in logging_event ---------------------------------------------------------
Key: LBCLASSIC-198 URL: http://jira.qos.ch/browse/LBCLASSIC-198 Project: logback-classic Issue Type: Improvement Components: appender Affects Versions: 0.9.18 Reporter: Michael Lynch Assignee: Logback dev list Attachments: DBAppender.java, DBAppenderBase.java
Want to retain longer history of logging events.
-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
_______________________________________________ logback-dev mailing list logback-dev@qos.ch http://qos.ch/mailman/listinfo/logback-dev

If I understand correctly, the event_id keyspace is partitioned among multiple databases which are synchronized with a central database. Thus, the partitioning is done by the databases and is transparent to DBAppender. I did not know you could partition the key space like that. On 22/03/2010 9:13 PM, Michael Lynch wrote:
It is mostly out of caution with some amount of fatalism mixed in.
We are working with a distributed databases that is synchronized back to a central database. In order to avoid key collisions we are partitioning the key space. We may have as many as 100 remote databases - reducing the key space for any one to ~21million.
While I don't expect 21 million errors from a single instance (nor even all together), there are processes that can run every few seconds that can generate an error. Left unchecked/unnoticed for long enough and the volume of errors could add up over time.
In my experience there is very little memory or disk space issues created by using a long instead of an int - although I recognize that logback is used in many different ways/environments, so my experience may not be as accurate in the context of logback.
- Michael

Yes - you have it right. We provide SQLAnywhere with the partition size and an identifier for each database - and it generates keys in the proper range. On Mon, Mar 22, 2010 at 1:22 PM, Ceki Gülcü <ceki@qos.ch> wrote:
If I understand correctly, the event_id keyspace is partitioned among multiple databases which are synchronized with a central database. Thus, the partitioning is done by the databases and is transparent to DBAppender. I did not know you could partition the key space like that.
On 22/03/2010 9:13 PM, Michael Lynch wrote:
It is mostly out of caution with some amount of fatalism mixed in.
We are working with a distributed databases that is synchronized back to a central database. In order to avoid key collisions we are partitioning the key space. We may have as many as 100 remote databases - reducing the key space for any one to ~21million.
While I don't expect 21 million errors from a single instance (nor even all together), there are processes that can run every few seconds that can generate an error. Left unchecked/unnoticed for long enough and the volume of errors could add up over time.
In my experience there is very little memory or disk space issues created by using a long instead of an int - although I recognize that logback is used in many different ways/environments, so my experience may not be as accurate in the context of logback.
- Michael
_______________________________________________ logback-dev mailing list logback-dev@qos.ch http://qos.ch/mailman/listinfo/logback-dev

[ http://jira.qos.ch/browse/LBCLASSIC-198?page=com.atlassian.jira.plugin.syste... ] Rü commented on LBCLASSIC-198: ------------------------------ Actually we produce on average about 1 GB of logs per day... for one application only! I haven't counted the actual log statements issued, but it's ~25 million lines. We don't throw any logs away, so it would not take too long for the 2 billion limit to be reached. And I don't think we are an absolute extreme, are we?
DBAppender: Support long event_id values in logging_event ---------------------------------------------------------
Key: LBCLASSIC-198 URL: http://jira.qos.ch/browse/LBCLASSIC-198 Project: logback-classic Issue Type: Improvement Components: appender Affects Versions: 0.9.18 Reporter: Michael Lynch Assignee: Logback dev list Attachments: DBAppender.java, DBAppenderBase.java
Want to retain longer history of logging events.
-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCLASSIC-198?page=com.atlassian.jira.plugin.syste... ] Joern Huxhorn commented on LBCLASSIC-198: ----------------------------------------- I'd also vote for 64bits here. If something really big is logging (e.g. Google) than 2 billion isn't really unrealistic. I'd always go for 64bit ids, simply to stay on the safe side.
DBAppender: Support long event_id values in logging_event ---------------------------------------------------------
Key: LBCLASSIC-198 URL: http://jira.qos.ch/browse/LBCLASSIC-198 Project: logback-classic Issue Type: Improvement Components: appender Affects Versions: 0.9.18 Reporter: Michael Lynch Assignee: Logback dev list Attachments: DBAppender.java, DBAppenderBase.java
Want to retain longer history of logging events.
-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCLASSIC-198?page=com.atlassian.jira.plugin.syste... ] Ceki Gulcu resolved LBCLASSIC-198. ---------------------------------- Resolution: Fixed Fixed in a recent commit.
DBAppender: Support long event_id values in logging_event ---------------------------------------------------------
Key: LBCLASSIC-198 URL: http://jira.qos.ch/browse/LBCLASSIC-198 Project: logback-classic Issue Type: Improvement Components: appender Affects Versions: 0.9.18 Reporter: Michael Lynch Assignee: Ceki Gulcu Attachments: DBAppender.java, DBAppenderBase.java
Want to retain longer history of logging events.
-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
participants (6)
-
Ceki Gulcu (JIRA)
-
Ceki Gülcü
-
Joern Huxhorn (JIRA)
-
Michael Lynch
-
Michael Lynch (JIRA)
-
Rü (JIRA)