[ http://jira.qos.ch/browse/LBCORE-147?page=com.atlassian.jira.plugin.system.i... ] Dieter Mueller commented on LBCORE-147: --------------------------------------- Same as for uncompressed old files. Files older than maxHistory are not deleted. Thanx in advance, Dieter Mueller

RollingPolicy MaxHistory not deleting old files -----------------------------------------------

Key: LBCORE-147 URL: http://jira.qos.ch/browse/LBCORE-147 Project: logback-core Issue Type: Bug Components: Rolling Affects Versions: 0.9.18 Environment: linux 2.6.18 64bits apache-tomcat-6.0.20 java/jre1.6.0_18 Reporter: Rafael Diaz Maurin Assignee: Logback dev list

The MaxHistory does not do the trick : the old files are not deleted at all : cat logging.xml <appender name="IDP_ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>/shibboleth-idp/logs/idp-access.log</File> <Append>false</Append> <ImmediateFlush>true</ImmediateFlush> <RollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>/shibboleth-idp/logs/old/idp-access-%d{yyyy-MM-dd_HH-mm}.log.gz</FileNamePattern> <MaxHistory>3</MaxHistory> </RollingPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%msg%n</Pattern> </layout> </appender>

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCORE-147?page=com.atlassian.jira.plugin.system.i... ] Ceki Gulcu commented on LBCORE-147: ----------------------------------- I am pretty confident that history clean up works rather well. Have you tried using more recent version of logback?

RollingPolicy MaxHistory not deleting old files -----------------------------------------------

Key: LBCORE-147 URL: http://jira.qos.ch/browse/LBCORE-147 Project: logback-core Issue Type: Bug Components: Rolling Affects Versions: 0.9.18 Environment: linux 2.6.18 64bits apache-tomcat-6.0.20 java/jre1.6.0_18 Reporter: Rafael Diaz Maurin Assignee: Ceki Gulcu

The MaxHistory does not do the trick : the old files are not deleted at all : cat logging.xml <appender name="IDP_ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>/shibboleth-idp/logs/idp-access.log</File> <Append>false</Append> <ImmediateFlush>true</ImmediateFlush> <RollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>/shibboleth-idp/logs/old/idp-access-%d{yyyy-MM-dd_HH-mm}.log.gz</FileNamePattern> <MaxHistory>3</MaxHistory> </RollingPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%msg%n</Pattern> </layout> </appender>

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCORE-147?page=com.atlassian.jira.plugin.system.i... ] Dieter Mueller commented on LBCORE-147: --------------------------------------- I wrote:

It seems, that logback calculates the ONE day to be deleted. It does not look back into the past, which is reasonable to me, because how far it should look back?

Perhaps a simple but unorthodox way would be, if there is a list (e.g. property-file) of created log files is maintained and persisted by logback. On every rollover the list is loaded and checked for old, deleteable files. I assume, the list would not be that big, so that there should be no big performance impact. One additional sugar piece would be, that if the filename patten (e.g. directory structure) is changed between a run of the application, logback will still be able, to delete an old logfile, which does not follow the new (changed) file name pattern. On the other hand, concurent access to this file via threads or several VMs will have to be synchronized? Dieter Mueller

RollingPolicy MaxHistory not deleting old files -----------------------------------------------

Key: LBCORE-147 URL: http://jira.qos.ch/browse/LBCORE-147 Project: logback-core Issue Type: Bug Components: Rolling Affects Versions: 0.9.18 Environment: linux 2.6.18 64bits apache-tomcat-6.0.20 java/jre1.6.0_18 Reporter: Rafael Diaz Maurin Assignee: Ceki Gulcu

The MaxHistory does not do the trick : the old files are not deleted at all : cat logging.xml <appender name="IDP_ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>/shibboleth-idp/logs/idp-access.log</File> <Append>false</Append> <ImmediateFlush>true</ImmediateFlush> <RollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>/shibboleth-idp/logs/old/idp-access-%d{yyyy-MM-dd_HH-mm}.log.gz</FileNamePattern> <MaxHistory>3</MaxHistory> </RollingPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%msg%n</Pattern> </layout> </appender>

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCORE-147?page=com.atlassian.jira.plugin.system.i... ] Eric Roberts commented on LBCORE-147: ------------------------------------- Referencing the configuration in Dieter Mueller's comment: <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <!-- rollover daily --> <fileNamePattern>foo-%d{yyyy-MM-dd}.%i.log.zip</fileNamePattern> <maxHistory>4</maxHistory> <timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP"> <maxFileSize>20MB</maxFileSize> </timeBasedFileNamingAndTriggeringPolicy> </rollingPolicy> I would expect this to only keep the last 4 days of log files. So if I had the following log files generated (assuming maxHistory=4 ): foo-2011-03-21.0.log.zip foo-2011-03-21.1.log.zip foo-2011-03-21.2.log.zip foo-2011-03-22.0.log.zip foo-2011-03-23.0.log.zip foo-2011-03-24.0.log.zip foo.log on 2011-03-26 I would expect that the logs from 2011-03-21 would be deleted and the listing would then be: foo-2011-03-22.0.log.zip foo-2011-03-23.0.log.zip foo-2011-03-24.0.log.zip foo-2011-03-25.0.log.zip foo.log However, this doesn't happen and I was unable to find any combination of rolling/triggering policies that would do this. It would seem that timeBasedFileNamingAndTriggeringPolicy ignores maxHistory.

RollingPolicy MaxHistory not deleting old files -----------------------------------------------

Key: LBCORE-147 URL: http://jira.qos.ch/browse/LBCORE-147 Project: logback-core Issue Type: Bug Components: Rolling Affects Versions: 0.9.18 Environment: linux 2.6.18 64bits apache-tomcat-6.0.20 java/jre1.6.0_18 Reporter: Rafael Diaz Maurin Assignee: Ceki Gulcu

The MaxHistory does not do the trick : the old files are not deleted at all : cat logging.xml <appender name="IDP_ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>/shibboleth-idp/logs/idp-access.log</File> <Append>false</Append> <ImmediateFlush>true</ImmediateFlush> <RollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>/shibboleth-idp/logs/old/idp-access-%d{yyyy-MM-dd_HH-mm}.log.gz</FileNamePattern> <MaxHistory>3</MaxHistory> </RollingPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%msg%n</Pattern> </layout> </appender>

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCORE-147?page=com.atlassian.jira.plugin.system.i... ] HC edited comment on LBCORE-147 at 7/1/11 4:35 PM: --------------------------------------------------- Hello, I can see the same issue on our production machines. We use logback and logs are not deleted using ch.qos.logback.core.rolling.TimeBasedRollingPolicy with ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP triggering policy. It seems issue can be replicated by slightly modifying one of the existing tests from logback's test suite: TimeBasedRollingWithArchiveRemovalTest.java. Please see the test named dailyChronologSizeBasedRolloverWhenLogFilenameDoesNotContainDirectory() in the attached file. It seems that if the log filename does not contain directory information no files are deleted. At a quick glance, the faulty code would be in SizeAndTimeBasedArchiveRemover.clean() 1. File parentDir = archive0.getParentFile(); 2. File[] matchingFileArray = FileFilterUtil.filesInFolderMatchingStemRegex(parentDir, stemRegex); The first line returns null for the specified case and hence FileFilterUtil.filesInFolderMatchingStemRegex() will return immediately when a null parameter is sent. Best Regards, Horace was (Author: hc): Hello, I can see the same issue on our production machines. We use logback and logs are not deleted using ch.qos.logback.core.rolling.TimeBasedRollingPolicy with ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP triggering policy. It seems issue can be replicated by slightly modifying one of the existing tests from logback's test suite: TimeBasedRollingWithArchiveRemovalTest.java. Please see the test named dailyChronologSizeBasedRolloverWhenLogFilenameDoesNotContainDirectory() in the attached file. It seems that if the log filename does not contain directory information no files are deleted. At a quick glance, the faulty code would be in {{SizeAndTimeBasedArchiveRemover.clean()}} {code} ... File parentDir = archive0.getParentFile(); File[] matchingFileArray = FileFilterUtil.filesInFolderMatchingStemRegex(parentDir, stemRegex); {code} {{File parentDir = archive0.getParentFile();}} will always return null and hence FileFilterUtil.filesInFolderMatchingStemRegex() will return immediately when a null parameter is sent. Best Regards, Horace

RollingPolicy MaxHistory not deleting old files -----------------------------------------------

Key: LBCORE-147 URL: http://jira.qos.ch/browse/LBCORE-147 Project: logback-core Issue Type: Bug Components: Rolling Affects Versions: 0.9.18 Environment: linux 2.6.18 64bits apache-tomcat-6.0.20 java/jre1.6.0_18 Reporter: Rafael Diaz Maurin Assignee: Ceki Gulcu Attachments: TimeBasedRollingWithArchiveRemovalTest.java

The MaxHistory does not do the trick : the old files are not deleted at all : cat logging.xml <appender name="IDP_ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>/shibboleth-idp/logs/idp-access.log</File> <Append>false</Append> <ImmediateFlush>true</ImmediateFlush> <RollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>/shibboleth-idp/logs/old/idp-access-%d{yyyy-MM-dd_HH-mm}.log.gz</FileNamePattern> <MaxHistory>3</MaxHistory> </RollingPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%msg%n</Pattern> </layout> </appender>

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCORE-147?page=com.atlassian.jira.plugin.system.i... ] HC updated LBCORE-147: ---------------------- Attachment: TimeBasedRollingWithArchiveRemovalTest.java

RollingPolicy MaxHistory not deleting old files -----------------------------------------------

Key: LBCORE-147 URL: http://jira.qos.ch/browse/LBCORE-147 Project: logback-core Issue Type: Bug Components: Rolling Affects Versions: 0.9.18 Environment: linux 2.6.18 64bits apache-tomcat-6.0.20 java/jre1.6.0_18 Reporter: Rafael Diaz Maurin Assignee: Ceki Gulcu Attachments: TimeBasedRollingWithArchiveRemovalTest.java

The MaxHistory does not do the trick : the old files are not deleted at all : cat logging.xml <appender name="IDP_ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>/shibboleth-idp/logs/idp-access.log</File> <Append>false</Append> <ImmediateFlush>true</ImmediateFlush> <RollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>/shibboleth-idp/logs/old/idp-access-%d{yyyy-MM-dd_HH-mm}.log.gz</FileNamePattern> <MaxHistory>3</MaxHistory> </RollingPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%msg%n</Pattern> </layout> </appender>

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCORE-147?page=com.atlassian.jira.plugin.system.i... ] Anton Kudinov commented on LBCORE-147: -------------------------------------- Have same issue on my production machines. In another production Iuse log4j with org.apache.log4j.appender.TimeAndSizeRollingAppender - it work very good

RollingPolicy MaxHistory not deleting old files -----------------------------------------------

Key: LBCORE-147 URL: http://jira.qos.ch/browse/LBCORE-147 Project: logback-core Issue Type: Bug Components: Rolling Affects Versions: 0.9.18 Environment: linux 2.6.18 64bits apache-tomcat-6.0.20 java/jre1.6.0_18 Reporter: Rafael Diaz Maurin Assignee: Ceki Gulcu Attachments: TimeBasedRollingWithArchiveRemovalTest.java

The MaxHistory does not do the trick : the old files are not deleted at all : cat logging.xml <appender name="IDP_ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>/shibboleth-idp/logs/idp-access.log</File> <Append>false</Append> <ImmediateFlush>true</ImmediateFlush> <RollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>/shibboleth-idp/logs/old/idp-access-%d{yyyy-MM-dd_HH-mm}.log.gz</FileNamePattern> <MaxHistory>3</MaxHistory> </RollingPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%msg%n</Pattern> </layout> </appender>

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCORE-147?page=com.atlassian.jira.plugin.system.i... ] Greg Thomas commented on LBCORE-147: ------------------------------------ You're right, in that I know think that implementing the "maxHistoryGap" solution I referred to earlier is the right solution. Instead, I think that when a new log file is created, rather than just delete the "oldest+1" file, it's necessary to do a full directory scan, and delete all those that are too old - that copes for longer quiet periods. Of course, it's complicated because your not just dealing with files called, e.g. "2011-02-12.log", but they could be called "2011/02/12/data.log" - i.e. it's necessary to do a recursive directory scan. Ceki; if you think that solution is acceptable, let me know and I'll see if I can work on a fix for it. PS. For those looking the other (working) appender mentioned in this ticket, org.apache.log4j.appender.TimeAndSizeRollingAppender is, despite the name, not part of the Apache project - you can find more details on this at http://www.simonsite.org.uk/ - from a quick glance, it too appears to "scavenge" older files on a configurable interval. I don't think that's necessary in this case; older files only need to be scavenged when a new one is created.

RollingPolicy MaxHistory not deleting old files -----------------------------------------------

Key: LBCORE-147 URL: http://jira.qos.ch/browse/LBCORE-147 Project: logback-core Issue Type: Bug Components: Rolling Affects Versions: 0.9.18 Environment: linux 2.6.18 64bits apache-tomcat-6.0.20 java/jre1.6.0_18 Reporter: Rafael Diaz Maurin Assignee: Ceki Gulcu Attachments: TimeBasedRollingWithArchiveRemovalTest.java

The MaxHistory does not do the trick : the old files are not deleted at all : cat logging.xml <appender name="IDP_ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>/shibboleth-idp/logs/idp-access.log</File> <Append>false</Append> <ImmediateFlush>true</ImmediateFlush> <RollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>/shibboleth-idp/logs/old/idp-access-%d{yyyy-MM-dd_HH-mm}.log.gz</FileNamePattern> <MaxHistory>3</MaxHistory> </RollingPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%msg%n</Pattern> </layout> </appender>

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCORE-147?page=com.atlassian.jira.plugin.system.i... ] Greg Thomas commented on LBCORE-147: ------------------------------------ I don't believe speed would be an issue; you could spawn off a scavenging thread at the time you create a new log file; however it would use more resources, which could be an issue on embedded systems. Though that can be minimised by putting files in a single directory.

RollingPolicy MaxHistory not deleting old files -----------------------------------------------

Key: LBCORE-147 URL: http://jira.qos.ch/browse/LBCORE-147 Project: logback-core Issue Type: Bug Components: Rolling Affects Versions: 0.9.18 Environment: linux 2.6.18 64bits apache-tomcat-6.0.20 java/jre1.6.0_18 Reporter: Rafael Diaz Maurin Assignee: Ceki Gulcu Attachments: TimeBasedRollingWithArchiveRemovalTest.java

The MaxHistory does not do the trick : the old files are not deleted at all : cat logging.xml <appender name="IDP_ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>/shibboleth-idp/logs/idp-access.log</File> <Append>false</Append> <ImmediateFlush>true</ImmediateFlush> <RollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>/shibboleth-idp/logs/old/idp-access-%d{yyyy-MM-dd_HH-mm}.log.gz</FileNamePattern> <MaxHistory>3</MaxHistory> </RollingPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%msg%n</Pattern> </layout> </appender>

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCORE-147?page=com.atlassian.jira.plugin.system.i... ] Greg Thomas commented on LBCORE-147: ------------------------------------ Some comments; 1. "it is sufficient to perform deletion of files in the said range only once, when the application starts" - what happens if the application is running but idle? Is there a thread that runs at midnight to rollover an empty file? (That's not what I'm seeing in logback 0.9.20, but I recognise I'm a bit behind on my versioning). Or does rollover only happen when a logging action occurs. If the latter, then you'll need to delete the files on application start *and* when a log file rolls over. However, in the latter case logback can know the value of Q, optimising things somewhat. 2. Perhaps make maxQ dependent on the rollover period, on the assumption that the more regularly you are rolling the more frequent you expect the logs to appear. e.g. something like If rolling monthly, maxQ = 1 year/12 files If rolling daily, maxQ = 60 days/60 files If rolling hourly, maxQ = 7 days/168 files 3. A variant on that - why not just make maxQ 60 rollovers. So ... If rolling monthly, maxQ = 60 months/5 years If rolling daily, maxQ = 60 days If rolling hourly, maxQ = 60 hours/2.5 days 4. Or make maxQ configurable - as I suggested on 21 Feb ("maxHistoryGap" is, I think the same as maxQ). 5. Although you suggest that a separate thread is overkill, checking/deleting multiple files *will* be slower than just deleting the one as currently occurs. I still think that doing it in a separate thread created at application start and each log file rollover would be better - to enable control to be returned to the application ASAP.

RollingPolicy MaxHistory not deleting old files -----------------------------------------------

Key: LBCORE-147 URL: http://jira.qos.ch/browse/LBCORE-147 Project: logback-core Issue Type: Bug Components: Rolling Affects Versions: 0.9.18 Environment: linux 2.6.18 64bits apache-tomcat-6.0.20 java/jre1.6.0_18 Reporter: Rafael Diaz Maurin Assignee: Ceki Gulcu Attachments: TimeBasedRollingWithArchiveRemovalTest.java

The MaxHistory does not do the trick : the old files are not deleted at all : cat logging.xml <appender name="IDP_ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>/shibboleth-idp/logs/idp-access.log</File> <Append>false</Append> <ImmediateFlush>true</ImmediateFlush> <RollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>/shibboleth-idp/logs/old/idp-access-%d{yyyy-MM-dd_HH-mm}.log.gz</FileNamePattern> <MaxHistory>3</MaxHistory> </RollingPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%msg%n</Pattern> </layout> </appender>

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCORE-147?page=com.atlassian.jira.plugin.system.i... ] Greg Thomas commented on LBCORE-147: ------------------------------------ 1. "Logback assumes that at least one LoggingEvent is generated per rollover period" That assumption does not hold for my application; it's a web application that is running 24/7. The web server may get restarted at very infrequent intervals (perhaps a few times a year, after much form filling), but the application is typically not used at the weekend - so for two days a week the application is running but there is no logging activity. I can't believe that's an especially unusual scenario. For example, an application running 24/7 may be highly used during the working day, necessitating a hourly rollover, but idle outside of the core hours. On the bright side, in this scenario at least logback can determine maxQ. 2. "the value of Q is independent of the length of the rollover period". It is, but I'd suggest that the value of maxQ, i.e. the max expected idle period, is a function of the rate of logging - which is often a function of the rate of rollover. Just trying to avoid the 1440 file scenario you highlighted.

RollingPolicy MaxHistory not deleting old files -----------------------------------------------

Key: LBCORE-147 URL: http://jira.qos.ch/browse/LBCORE-147 Project: logback-core Issue Type: Bug Components: Rolling Affects Versions: 0.9.18 Environment: linux 2.6.18 64bits apache-tomcat-6.0.20 java/jre1.6.0_18 Reporter: Rafael Diaz Maurin Assignee: Ceki Gulcu Attachments: TimeBasedRollingWithArchiveRemovalTest.java

The MaxHistory does not do the trick : the old files are not deleted at all : cat logging.xml <appender name="IDP_ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>/shibboleth-idp/logs/idp-access.log</File> <Append>false</Append> <ImmediateFlush>true</ImmediateFlush> <RollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>/shibboleth-idp/logs/old/idp-access-%d{yyyy-MM-dd_HH-mm}.log.gz</FileNamePattern> <MaxHistory>3</MaxHistory> </RollingPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%msg%n</Pattern> </layout> </appender>

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCORE-147?page=com.atlassian.jira.plugin.system.i... ] Joern Huxhorn commented on LBCORE-147: -------------------------------------- What about appenders that are configured to only log errors? I wouldn't want to log an error event as a heartbeat just to get the rolling right.

RollingPolicy MaxHistory not deleting old files -----------------------------------------------

Key: LBCORE-147 URL: http://jira.qos.ch/browse/LBCORE-147 Project: logback-core Issue Type: Bug Components: Rolling Affects Versions: 0.9.18 Environment: linux 2.6.18 64bits apache-tomcat-6.0.20 java/jre1.6.0_18 Reporter: Rafael Diaz Maurin Assignee: Ceki Gulcu Attachments: TimeBasedRollingWithArchiveRemovalTest.java

The MaxHistory does not do the trick : the old files are not deleted at all : cat logging.xml <appender name="IDP_ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>/shibboleth-idp/logs/idp-access.log</File> <Append>false</Append> <ImmediateFlush>true</ImmediateFlush> <RollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>/shibboleth-idp/logs/old/idp-access-%d{yyyy-MM-dd_HH-mm}.log.gz</FileNamePattern> <MaxHistory>3</MaxHistory> </RollingPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%msg%n</Pattern> </layout> </appender>

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCORE-147?page=com.atlassian.jira.plugin.system.i... ] Ceki Gulcu commented on LBCORE-147: ----------------------------------- For applications without logging activity for a prolonged time, it would be quite possible to detect such inactivity and react accordingly without requiring a heart beat.

RollingPolicy MaxHistory not deleting old files -----------------------------------------------

Key: LBCORE-147 URL: http://jira.qos.ch/browse/LBCORE-147 Project: logback-core Issue Type: Bug Components: Rolling Affects Versions: 0.9.18 Environment: linux 2.6.18 64bits apache-tomcat-6.0.20 java/jre1.6.0_18 Reporter: Rafael Diaz Maurin Assignee: Ceki Gulcu Attachments: TimeBasedRollingWithArchiveRemovalTest.java

The MaxHistory does not do the trick : the old files are not deleted at all : cat logging.xml <appender name="IDP_ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>/shibboleth-idp/logs/idp-access.log</File> <Append>false</Append> <ImmediateFlush>true</ImmediateFlush> <RollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>/shibboleth-idp/logs/old/idp-access-%d{yyyy-MM-dd_HH-mm}.log.gz</FileNamePattern> <MaxHistory>3</MaxHistory> </RollingPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%msg%n</Pattern> </layout> </appender>

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCORE-147?page=com.atlassian.jira.plugin.system.i... ] Greg Thomas commented on LBCORE-147: ------------------------------------ I'm still worried about relying on the application to log frequently; Unless I've missed something ... a) It means two different file deletion mechanisms to maintain within LBCORE; one at startup, one at rollover time. b) The application will have to parse the logging configuration, and determine the current logging level and frequency to determine if a new log entry has of sufficient level has occurred sufficiently recently, and then create a "please ignore this" log entry at a high enough level to trigger a new rollover. c) Any application checking for log entries to alarm will need to be altered to ignore the "please ignore this" log entry.

RollingPolicy MaxHistory not deleting old files -----------------------------------------------

Key: LBCORE-147 URL: http://jira.qos.ch/browse/LBCORE-147 Project: logback-core Issue Type: Bug Components: Rolling Affects Versions: 0.9.18 Environment: linux 2.6.18 64bits apache-tomcat-6.0.20 java/jre1.6.0_18 Reporter: Rafael Diaz Maurin Assignee: Ceki Gulcu Attachments: TimeBasedRollingWithArchiveRemovalTest.java

The MaxHistory does not do the trick : the old files are not deleted at all : cat logging.xml <appender name="IDP_ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>/shibboleth-idp/logs/idp-access.log</File> <Append>false</Append> <ImmediateFlush>true</ImmediateFlush> <RollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>/shibboleth-idp/logs/old/idp-access-%d{yyyy-MM-dd_HH-mm}.log.gz</FileNamePattern> <MaxHistory>3</MaxHistory> </RollingPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%msg%n</Pattern> </layout> </appender>

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCORE-147?page=com.atlassian.jira.plugin.system.i... ] Ceki Gulcu commented on LBCORE-147: ----------------------------------- The archive removal algorithm now [1] properly deals with stop/restarts as long as the time difference is not over 64 days. For applications not generating logs, the (logging) inactivity period can be arbitrarily long. As long as there is a single log activity archives will be removed correctly. [1] http://github.com/ceki/logback/commit/524e0fc13ecc4fb24d

RollingPolicy MaxHistory not deleting old files -----------------------------------------------

Key: LBCORE-147 URL: http://jira.qos.ch/browse/LBCORE-147 Project: logback-core Issue Type: Bug Components: Rolling Affects Versions: 0.9.18 Environment: linux 2.6.18 64bits apache-tomcat-6.0.20 java/jre1.6.0_18 Reporter: Rafael Diaz Maurin Assignee: Ceki Gulcu Attachments: TimeBasedRollingWithArchiveRemovalTest.java

The MaxHistory does not do the trick : the old files are not deleted at all : cat logging.xml <appender name="IDP_ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>/shibboleth-idp/logs/idp-access.log</File> <Append>false</Append> <ImmediateFlush>true</ImmediateFlush> <RollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>/shibboleth-idp/logs/old/idp-access-%d{yyyy-MM-dd_HH-mm}.log.gz</FileNamePattern> <MaxHistory>3</MaxHistory> </RollingPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%msg%n</Pattern> </layout> </appender>

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCORE-147?page=com.atlassian.jira.plugin.system.i... ] Bruce E. Irving commented on LBCORE-147: ---------------------------------------- I have downloaded Logback version 0.9.30 which is supposed to fix this issue; but I have a usage scenario for which this is still broken. The application is a desktop utility which users may run several times during the day or it may go unused for long periods. I wanted to collect the logs for a given day in a date stamped file and to save a few days and automatically delete older files. For the datestamp requirement, the FileAppender would work, but to delete older files the RollingFileAppender seemed appropriate. I am not compressing old files, so that is not an issue. I am not providing the FileAppender.file property, so each days logging goes directly into the file with today's date, and there is no need for file renaming. As far as I can tell, the 'rolling' logic that would delete the old files is never run. The application is almost never running at midnight, so it would have run at startup and does not. I found what I believe is the source of the issue in TimeBasedFileNamingAndTriggeringPolicyBase: public void start() { ... setDateInCurrentPeriod(new Date(getCurrentTime())); if (tbrp.getParentsRawFileProperty() != null) { File currentFile = new File(tbrp.getParentsRawFileProperty()); if (currentFile.exists() && currentFile.canRead()) { setDateInCurrentPeriod(new Date(currentFile.lastModified())); } } addInfo("Setting initial period to " + dateInCurrentPeriod); computeNextCheck(); } Since there is no parentsRawFileProperty, this uses the current time and sets the next check at midnight tonight. Therefore the check is not run at startup, and in my case is never run. This would work fine if dateInCurrentPeriod was set to anything before today. I would suggest the following change: public void start() { ... File currentFile = new File(tbrp.getActiveFileName()); // Use same logic if ParentsRawFileProperty is set or not. if (currentFile.exists() && currentFile.canRead()) { setDateInCurrentPeriod(new Date(currentFile.lastModified())); } else { setDateInCurrentPeriod(0); // No current file found, force a triggering event // or if the above is too brute-force: setDateInCurrentPeriod(rc.getRelativeDate(new Date(), -1)); } addInfo("Setting initial period to " + dateInCurrentPeriod); computeNextCheck(); }

RollingPolicy MaxHistory not deleting old files -----------------------------------------------

Key: LBCORE-147 URL: http://jira.qos.ch/browse/LBCORE-147 Project: logback-core Issue Type: Bug Components: Rolling Affects Versions: 0.9.18 Environment: linux 2.6.18 64bits apache-tomcat-6.0.20 java/jre1.6.0_18 Reporter: Rafael Diaz Maurin Assignee: Ceki Gulcu Fix For: 0.9.30

Attachments: TimeBasedRollingWithArchiveRemovalTest.java

The MaxHistory does not do the trick : the old files are not deleted at all : cat logging.xml <appender name="IDP_ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>/shibboleth-idp/logs/idp-access.log</File> <Append>false</Append> <ImmediateFlush>true</ImmediateFlush> <RollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>/shibboleth-idp/logs/old/idp-access-%d{yyyy-MM-dd_HH-mm}.log.gz</FileNamePattern> <MaxHistory>3</MaxHistory> </RollingPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%msg%n</Pattern> </layout> </appender>

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCORE-147?page=com.atlassian.jira.plugin.system.i... ] Shaun thompson commented on LBCORE-147: --------------------------------------- I'm currently referencing the latest logback 0.9.30 version and I'm not seeing my log history deleted correctly. Possible configuration issue? Here's my log policy <appender name="file" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>XXX.log</file> <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <!-- rollover daily --> <fileNamePattern>XXX-%d{yyyy-MM-dd}.%i.log</fileNamePattern> <MaxHistory>7</MaxHistory> <timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP"> <maxFileSize>5MB</maxFileSize> <MaxHistory>3</MaxHistory> </timeBasedFileNamingAndTriggeringPolicy> </rollingPolicy> <encoder class="CustomPatternLayoutEncoder"> <pattern>[%p] [%d{ISO8601}] [%ip_address] [%user_id] [%c] [%m]%n</pattern> </encoder> </appender> Here's a history of the log files that are still out there. XXX-2011-08-26.0.log XXX-2011-08-29.0.log XXX-2011-08-30.0.log XXX-2011-09-09.0.log XXX-2011-09-16.0.log XXX-2011-09-23.0.log XXX-2011-09-27.0.log XXX-2011-09-28.0.log XXX-2011-09-29.0.log XXX-2011-09-30.0.log XXX-2011-10-02.0.log XXX-2011-10-03.0.log

RollingPolicy MaxHistory not deleting old files -----------------------------------------------

Key: LBCORE-147 URL: http://jira.qos.ch/browse/LBCORE-147 Project: logback-core Issue Type: Bug Components: Rolling Affects Versions: 0.9.18 Environment: linux 2.6.18 64bits apache-tomcat-6.0.20 java/jre1.6.0_18 Reporter: Rafael Diaz Maurin Assignee: Ceki Gulcu Fix For: 0.9.30

Attachments: TimeBasedRollingWithArchiveRemovalTest.java

The MaxHistory does not do the trick : the old files are not deleted at all : cat logging.xml <appender name="IDP_ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>/shibboleth-idp/logs/idp-access.log</File> <Append>false</Append> <ImmediateFlush>true</ImmediateFlush> <RollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>/shibboleth-idp/logs/old/idp-access-%d{yyyy-MM-dd_HH-mm}.log.gz</FileNamePattern> <MaxHistory>3</MaxHistory> </RollingPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%msg%n</Pattern> </layout> </appender>

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.qos.ch/browse/LBCORE-147?page=com.atlassian.jira.plugin.system.i... ] Shaun thompson commented on LBCORE-147: --------------------------------------- That could definitely be a possibility. The logs I'm looking at are for a development application so the start/stop frequency varies widely - anywhere from several times a day to once in a couple of weeks.

RollingPolicy MaxHistory not deleting old files -----------------------------------------------

Key: LBCORE-147 URL: http://jira.qos.ch/browse/LBCORE-147 Project: logback-core Issue Type: Bug Components: Rolling Affects Versions: 0.9.18 Environment: linux 2.6.18 64bits apache-tomcat-6.0.20 java/jre1.6.0_18 Reporter: Rafael Diaz Maurin Assignee: Ceki Gulcu Fix For: 0.9.30

Attachments: TimeBasedRollingWithArchiveRemovalTest.java

The MaxHistory does not do the trick : the old files are not deleted at all : cat logging.xml <appender name="IDP_ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>/shibboleth-idp/logs/idp-access.log</File> <Append>false</Append> <ImmediateFlush>true</ImmediateFlush> <RollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>/shibboleth-idp/logs/old/idp-access-%d{yyyy-MM-dd_HH-mm}.log.gz</FileNamePattern> <MaxHistory>3</MaxHistory> </RollingPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%msg%n</Pattern> </layout> </appender>

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.qos.ch/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira