Hello, all. First let me introduce myself. I'm Leonardo Lima, and I've been using log4j for about 2 years. In the past, I needed to generate one log file per 'context', context being anything that I put in my MDC. I went to the log4j user list, discussed a little with other users and I used an implementation that I found at log4j bug tracking system. I've been using it since October last year and it's all ok. Sebastien contacted me and the other guys that were talking about (how to) implementing what became the MultiFileAppender so that we could contribute it to logback; but at the time I couldn't switch to nor learn logback. Now I want to use logback and reproduce the functionality of MultiFileAppender (that I went and improved myself over). So, the result's in the emails that follows. So, please, any input/feedback is much welcome. I improved the code below using EHCache as caching solution to handle the timeout/max open files problems that would occour, but maybe something simpler and built-in is better. And, as an improvement, I thought about applying the whole PatternLayout parsing to filenames, so we could use thread names and much more information on file names. That and the option to auto-create dirs, would be awesome (at least for me). Think about self-organizing log! Thanks a lot for your time. Leonardo de Moura Rocha Lima -----Mensagem original----- De: Ceki Gulcu [mailto:ceki@qos.ch] Enviada em: terça-feira, 29 de maio de 2007 15:25 Para: Leonardo de Moura Rocha Lima Cc: 'Sebastien Pennec' Assunto: Re: RES: Implementing MultiFileAppender Hello Leonardo, Since this conversation might interest other users, could you please post your message on the logback-user mailing list? Many thanks, Leonardo de Moura Rocha Lima wrote:
Hello, Sebastien.
I'm sorry that it took such a long time to reply your e-mail, but only now I got 'green light' to go into logback. We now have a new project that we'd like to use logback and keep the MultiFileAppender behavior (developed in-house) that we had in log4j. For what I gathered, the concept of using MDC properties in filenames isn't yet implement in logback. So, I am on my way to implement it.
I tried for a couple of hours to understand and use your recommendation about using Joran replay functionality to achieve what we need, but failed. I did run and (I think I) understood the FruitConfigurationTest. I read Chapter 3 of the logback Manual that has an example of using Joran to re/load configuration. But I failed to understand how I replay the configuration once the first run is running.
So I went and did something that's not that beautiful, but works. I implemented an Appender has a parser that substitutes the %X{} and %d{} tags (not recursively, but more than one tag per filename. Example: client.%X{ip}.%X{port}.%d{yyyy}.%d{MM}.log) and create a new filename from a base filename, plus a map of FileAppenders to delegate the logging events to. It's a naïve approach, but it's suiting my needs (use MDC to generate context, roll logger using Time). Only thing left is handling appenders 'timeout' and 'max open files', so I don't get too many file handles nor keep file handles open for no reason. I thought about using some cache solution to handle that, but it would add more dependency to the project...
I'd like to request your guidance on the matter. Should I go on with what I wrote (with or without the cache), or please teach me how the 'replay' functionality would be used in this matter. And, if I wanted to contribute the code, what should I provide. I've never contributed anything before...
I embed the code of my class here because I don't know how you handle attachments...
Thanks in advance, and keep up the good work on logback.
Leonardo de Moura Rocha Lima 55 11 3094.3939
import java.text.SimpleDateFormat; import java.util.Date; import java.util.HashMap; import java.util.Map;
import ch.qos.logback.classic.spi.LoggingEvent; import ch.qos.logback.core.AppenderBase; import ch.qos.logback.core.FileAppender; import ch.qos.logback.core.Layout;
public class LAppender extends AppenderBase<LoggingEvent> {
/** * Base Filename. Will be parsed for %X{} and %d{} tags. */ private String file;
/** * FileAppender option; will propagate. */ private Layout<LoggingEvent> layout;
/** * FileAppender option; will propagate. */ private boolean immediateFlush = true;
/** * FileAppender option; will propagate. */ private boolean bufferedIO;
/** * FileAppender option; will propagate. */ private String encoding;
/** * FileAppender option; will propagate. */ private boolean append;
/** * FileAppender option; will propagate. */ private int bufferSize;
/** * Map of current active FileAppenders. Needs improvement: max open files * and timeout, to keep resources */ private Map<String, FileAppender<LoggingEvent>> appenders = new HashMap<String, FileAppender<LoggingEvent>>();
/** * Return the FileAppender for the given filename. Create a new one and add * to cache, if needed. */ private FileAppender<LoggingEvent> getFA(String filename) { if (appenders.get(filename) != null) { return appenders.get(filename); }
FileAppender<LoggingEvent> resp = new FileAppender<LoggingEvent>(); resp.setAppend(append); resp.setBufferedIO(bufferedIO); resp.setBufferSize(bufferSize); resp.setContext(this.getContext()); resp.setEncoding(encoding); resp.setFile(filename); resp.setImmediateFlush(immediateFlush); resp.setLayout(layout); resp.setName(this.getName()); resp.start(); appenders.put(filename, resp); return resp; }
@Override /** * Delegate the LoggingEvent to the right FileAppender */ protected void append(LoggingEvent eventObject) { getFA(getFile(eventObject)).doAppend(eventObject); }
/** * Return the filename for the given LoggingEvent, based on the base * filename. */ private String getFile(LoggingEvent eventObject) { return substituteDate(substituteContext(file, eventObject .getMDCPropertyMap()), new Date(eventObject.getTimeStamp())); }
/** * Parse and replace %X{} tags. */ private String substituteContext(String base, Map<String, String> mdc) { if (base == null) return null; if (mdc == null) return base;
int idx = base.indexOf("%X{"); if (idx < 0) return base; int lidx = base.indexOf('}', idx); String str = base.substring(idx + 3, lidx); String val = mdc.get(str); if (val == null) val = "null"; return substituteContext(base.replaceAll("\\%X\\{" + str + '}', val), mdc); }
/** * Parse and replace %d{} tags. */ private String substituteDate(String base, Date dt) { if (base == null) return null; if (dt == null) dt = new Date();
int idx = base.indexOf("%d{"); if (idx < 0) return base; int lidx = base.indexOf('}', idx); String str = base.substring(idx + 3, lidx); String val = new SimpleDateFormat(str).format(dt); if (val == null) val = "null"; return substituteDate(base.replaceAll("\\%d\\{" + str + '}', val), dt); }
@Override /** * Propagate the stop action to everyone */ public void stop() { for (FileAppender<LoggingEvent> fa : this.appenders.values()) { fa.stop(); } }
public void setFile(String file) { this.file = file; }
public void setLayout(Layout<LoggingEvent> layout) { this.layout = layout; }
public void setAppend(boolean append) { this.append = append; }
public void setBufferedIO(boolean bufferedIO) { this.bufferedIO = bufferedIO; }
public void setEncoding(String encoding) { this.encoding = encoding; }
public void setImmediateFlush(boolean immediateFlush) { this.immediateFlush = immediateFlush; }
public void setBufferSize(int bufferSize) { this.bufferSize = bufferSize; }
}
-----Mensagem original----- De: Sebastien Pennec [mailto:sebastien@qos.ch] Enviada em: segunda-feira, 23 de outubro de 2006 14:53 Para: HBender@Ergonomics.ch; Patrick.Wyss@mobilesolutions.ch; online@l3o.net Cc: Ceki Gülcü Assunto: Implementing MultiFileAppender
Hello Heri and Leo and Patrick,
You might have heard of logback, a rewrite of log4j by Ceki Gülcü, log4j's founder. Logback has a smaller memory footprint and is faster by a wide margin. It has much better error reporting capability and is generally more robust. Logback has support for markers which enables capabilities not found elsewhere. We will be documenting these capabilities in the near future.
We've read your mails on the log4j mailing list about a MultiFileAppender, and found your idea very interesting.
In logback, there is mostly all the bricks one might need to create such a component.
Our new version of the Joran configuration framework allows replay of configuration elements. This is exactly what's needed to create new FileAppenders based on specific situations. When replaying configuration sequences, one can replace some properties, such as file name, to fine tune the new component.
For documentation about Joran, you might want to check the following resources
- http://logback.qos.ch/joran.html - logback-core/examples/src/joran/ (available from Subversion repo) - ch.qos.logback.core.joran.replay.FruitConfigurationTest (Subversion repo)
Replay functionality in Joran is only available from the Subversion repo. It will be included in logback version 0.5.
Regarding FileAppender and derived appenders, logback has similar support compared to log4j 1.3, in addition to several niceties.
Would you like to contribute MultiFileAppender to logback? If you are interested, we will gladly provide support for this endeavor.
Kind regards,
Sébastien
-- Ceki Gülcü Logback: The reliable, generic, fast and flexible logging framework for Java. http://logback.qos.ch