Logback appender for MongoDB
Hello logback developers/experts, I recently started writing a Logback appender that writes/appends logs to MongoDB: https://github.com/belaso/logback-mongodb Would greatly appreciate your feedback on: 1) is a MongoDB appender a good idea? 2) if yes, which is the best way to promote this project to Logback community? Regards Christian Trutz
I wrote one myself some time ago [1], [2] and there is an issue open [3]. I'm sure I saw few other implementations as well. Looks like the best place for MongoDB appender is logback-extensions project. We should merge ideas from all available implementations into one, stable, feature rich implementation. [1]: https://github.com/nurkiewicz/logback/tree/mongodb-appender [2]: http://nurkiewicz.blogspot.no/2011/04/mongodb-and-recording-appenders-for.ht... [3]: http://jira.qos.ch/browse/LOGBACK-522 [4]: https://github.com/qos-ch/logback-extensions On 13.06.2012 16:33, Christian Trutz wrote:
Hello logback developers/experts,
I recently started writing a Logback appender that writes/appends logs to MongoDB:
https://github.com/belaso/logback-mongodb
Would greatly appreciate your feedback on:
1) is a MongoDB appender a good idea?
2) if yes, which is the best way to promote this project to Logback community?
Regards Christian Trutz
_______________________________________________ logback-dev mailing list logback-dev@qos.ch http://mailman.qos.ch/mailman/listinfo/logback-dev
pozdrowienia -- Tomasz Nurkiewicz http://nurkiewicz.blogspot.com
Hi,
We should merge ideas from all available implementations into one, stable, feature rich implementation.
+1 a very good idea :-)
Looks like the best place for MongoDB appender is logback-extensions project.
What is the contribution policy on logback-extensions? I see only: https://github.com/qos-ch/logback-extensions/wiki/Developers Can you grant commiter privs to me (https://github.com/belaso) on logback-extension? I am an experienced OSS contributor (e.g. Eclipse EGit, see also http://www.ohloh.net/p/egit/contributors?page=2 ) I will cherry pick ideas from other implementations (mine also) and consolidate this to log back-extensions ... Regards Christian 2012/6/13 Tomasz Nurkiewicz <nurkiewicz@gmail.com>
I wrote one myself some time ago [1], [2] and there is an issue open [3]. I'm sure I saw few other implementations as well.
Looks like the best place for MongoDB appender is logback-extensions project. We should merge ideas from all available implementations into one, stable, feature rich implementation.
[1]: https://github.com/nurkiewicz/**logback/tree/mongodb-appender<https://github.com/nurkiewicz/logback/tree/mongodb-appender> [2]: http://nurkiewicz.blogspot.no/**2011/04/mongodb-and-recording-** appenders-for.html<http://nurkiewicz.blogspot.no/2011/04/mongodb-and-recording-appenders-for.html> [3]: http://jira.qos.ch/browse/**LOGBACK-522<http://jira.qos.ch/browse/LOGBACK-522> [4]: https://github.com/qos-ch/**logback-extensions<https://github.com/qos-ch/logback-extensions>
On 13.06.2012 16:33, Christian Trutz wrote:
Hello logback developers/experts,
I recently started writing a Logback appender that writes/appends logs to MongoDB:
https://github.com/belaso/**logback-mongodb<https://github.com/belaso/logback-mongodb>
Would greatly appreciate your feedback on:
1) is a MongoDB appender a good idea?
2) if yes, which is the best way to promote this project to Logback community?
Regards Christian Trutz
______________________________**_________________ logback-dev mailing list logback-dev@qos.ch http://mailman.qos.ch/mailman/**listinfo/logback-dev<http://mailman.qos.ch/mailman/listinfo/logback-dev>
pozdrowienia
-- Tomasz Nurkiewicz http://nurkiewicz.blogspot.com ______________________________**_________________ logback-dev mailing list logback-dev@qos.ch http://mailman.qos.ch/mailman/**listinfo/logback-dev<http://mailman.qos.ch/mailman/listinfo/logback-dev>
-- Christian Trutz Von-Flotow-Straße 24 D-45772 Marl Festnetz (privat): +49 (0)2365 3840327 E-Mail: christian.trutz@gmail.com
Hi Christian, Tomasz created a jira issue for implementing MongoDBAppender [2]. His implementation can be found at [3, 4, 5]. I think his impl is worth a look. He seprated connection functionality into MongoDBAppenderBase (part of logback-core) and ILoggingEvent specific code into MongoDBAppender (in logback-classic). Moreover, his code comes with 100% test coverage! I guess the code could be easily extended to cater for AccessEvent (in logback-access). Anyway, let me know what I can do to move this work forward. [1] https://github.com/qos-ch/logback-extensions [2] http://jira.qos.ch/browse/LOGBACK-522 [3] https://github.com/nurkiewicz/logback/tree/mongodb-appender [4] https://github.com/nurkiewicz/logback/commit/858aceb9af74d0 [5] https://github.com/nurkiewicz/logback/commit/f5bff8e43 -- Ceki http://twitter.com/#!/ceki On 14.06.2012 00:36, Christian Trutz wrote:
Hi,
We should merge ideas from all available implementations into one, stable, feature rich implementation.
+1 a very good idea :-)
Looks like the best place for MongoDB appender is logback-extensions project.
What is the contribution policy on logback-extensions? I see only: https://github.com/qos-ch/logback-extensions/wiki/Developers
Can you grant commiter privs to me (https://github.com/belaso) on logback-extension? I am an experienced OSS contributor (e.g. Eclipse EGit, see also http://www.ohloh.net/p/egit/contributors?page=2 ) I will cherry pick ideas from other implementations (mine also) and consolidate this to log back-extensions ...
Regards Christian
Here is precisely what I created so far: * base abstract appender handling MongoDB connectivity [1] * standard appender [2] + tests [3] * access appender (tested with Tomcat) [4] + tests [5] * configuration examples [6] By quick googling I found several other community implementations: [7], [8], [9]. Looks like my implementation is the only one capable of storing access logs as well, e.g.: { "_id" : ObjectId("4d98cc4f7abb95e59279e183"), "timeStamp" : ISODate("2011-04-03T19:36:47.339Z"), "server" : "localhost", "remote" : { "host" : "0:0:0:0:0:0:0:1", "user" : "tomcat", "addr" : "0:0:0:0:0:0:0:1" }, "request" : { "uri" : "/manager/images/tomcat.gif", "protocol" : "HTTP/1.1", "method" : "GET", "sessionId" : "1C6357816D9EEFD31F6D9D154D87308A", "userAgent" : "Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.2.16) Gecko/20110323 Ubuntu/10.10 (maverick) Firefox/3.6.16", "referer" : "http://localhost:8080/manager/html" }, "response" : { "contentLength" : NumberLong(1934), "statusCode" : 200 } } We should now revise all available implementations and build final one based on best ideas from all of them. The only issue I see is integration testing: at least my tests require running MongoDB instance somewhere. This is a major drawback for both continuous integration of Lobgack extensions and for all developers, not necessarily interested in this particular extension. Christian Trutz - thanks for creating the project structure. [1]: https://github.com/nurkiewicz/logback/blob/mongodb-appender/logback-core/src... [2]: https://github.com/nurkiewicz/logback/blob/mongodb-appender/logback-classic/... [3]: https://github.com/nurkiewicz/logback/blob/mongodb-appender/logback-classic/... [4]: https://github.com/nurkiewicz/logback/blob/mongodb-appender/logback-access/s... [5]: https://github.com/nurkiewicz/logback/blob/mongodb-appender/logback-access/s... [6]: https://github.com/nurkiewicz/logback/tree/mongodb-appender/logback-classic/... [7]: https://github.com/kofemann/mongo-appender-for-logback [8]: https://github.com/smilebase/logback-mongodb [9]: https://github.com/uded/logback-mongodb-appender On 14.06.2012 14:24, ceki wrote:
Hi Christian,
Tomasz created a jira issue for implementing MongoDBAppender [2]. His implementation can be found at [3, 4, 5].
I think his impl is worth a look. He seprated connection functionality into MongoDBAppenderBase (part of logback-core) and ILoggingEvent specific code into MongoDBAppender (in logback-classic). Moreover, his code comes with 100% test coverage!
I guess the code could be easily extended to cater for AccessEvent (in logback-access).
Anyway, let me know what I can do to move this work forward.
[1] https://github.com/qos-ch/logback-extensions [2] http://jira.qos.ch/browse/LOGBACK-522 [3] https://github.com/nurkiewicz/logback/tree/mongodb-appender [4] https://github.com/nurkiewicz/logback/commit/858aceb9af74d0 [5] https://github.com/nurkiewicz/logback/commit/f5bff8e43
pozdrowienia -- Tomasz Nurkiewicz http://nurkiewicz.blogspot.com
Hi Tomasz,
The only issue I see is integration testing: at least my tests require running MongoDB instance somewhere. This is a major drawback for both continuous integration of Lobgack extensions and for all developers, not necessarily interested in this particular extension.
I think we should not require for unit test a mongodb instance. I have very good experience with jmockit http://code.google.com/p/jmockit/ what do you think about? Christian 2012/6/14 Tomasz Nurkiewicz <nurkiewicz@gmail.com>
Here is precisely what I created so far:
* base abstract appender handling MongoDB connectivity [1] * standard appender [2] + tests [3] * access appender (tested with Tomcat) [4] + tests [5] * configuration examples [6]
By quick googling I found several other community implementations: [7], [8], [9]. Looks like my implementation is the only one capable of storing access logs as well, e.g.:
{ "_id" : ObjectId("**4d98cc4f7abb95e59279e183"), "timeStamp" : ISODate("2011-04-03T19:36:47.**339Z"), "server" : "localhost", "remote" : { "host" : "0:0:0:0:0:0:0:1", "user" : "tomcat", "addr" : "0:0:0:0:0:0:0:1" }, "request" : { "uri" : "/manager/images/tomcat.gif", "protocol" : "HTTP/1.1", "method" : "GET", "sessionId" : "**1C6357816D9EEFD31F6D9D154D8730**8A", "userAgent" : "Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.2.16) Gecko/20110323 Ubuntu/10.10 (maverick) Firefox/3.6.16", "referer" : "http://localhost:8080/**manager/html<http://localhost:8080/manager/html> " }, "response" : { "contentLength" : NumberLong(1934), "statusCode" : 200 } }
We should now revise all available implementations and build final one based on best ideas from all of them. The only issue I see is integration testing: at least my tests require running MongoDB instance somewhere. This is a major drawback for both continuous integration of Lobgack extensions and for all developers, not necessarily interested in this particular extension.
Christian Trutz - thanks for creating the project structure.
[1]: https://github.com/nurkiewicz/**logback/blob/mongodb-appender/** logback-core/src/main/java/ch/**qos/logback/core/db/mongo/** MongoDBAppenderBase.java<https://github.com/nurkiewicz/logback/blob/mongodb-appender/logback-core/src/main/java/ch/qos/logback/core/db/mongo/MongoDBAppenderBase.java>
[2]: https://github.com/nurkiewicz/**logback/blob/mongodb-appender/** logback-classic/src/main/java/**ch/qos/logback/classic/db/** mongo/MongoDBAppender.java<https://github.com/nurkiewicz/logback/blob/mongodb-appender/logback-classic/src/main/java/ch/qos/logback/classic/db/mongo/MongoDBAppender.java>
[3]: https://github.com/nurkiewicz/**logback/blob/mongodb-appender/** logback-classic/src/test/java/**ch/qos/logback/classic/db/** mongo/MongoDBAppenderTest.java<https://github.com/nurkiewicz/logback/blob/mongodb-appender/logback-classic/src/test/java/ch/qos/logback/classic/db/mongo/MongoDBAppenderTest.java>
[4]: https://github.com/nurkiewicz/**logback/blob/mongodb-appender/** logback-access/src/main/java/**ch/qos/logback/access/db/** mongo/MongoDBAppender.java<https://github.com/nurkiewicz/logback/blob/mongodb-appender/logback-access/src/main/java/ch/qos/logback/access/db/mongo/MongoDBAppender.java>
[5]: https://github.com/nurkiewicz/**logback/blob/mongodb-appender/** logback-access/src/test/java/**ch/qos/logback/access/db/** mongo/MongoDBAppenderTest.java<https://github.com/nurkiewicz/logback/blob/mongodb-appender/logback-access/src/test/java/ch/qos/logback/access/db/mongo/MongoDBAppenderTest.java>
[6]: https://github.com/nurkiewicz/**logback/tree/mongodb-appender/** logback-classic/src/test/**input/joran/mongodb<https://github.com/nurkiewicz/logback/tree/mongodb-appender/logback-classic/src/test/input/joran/mongodb>
[8]: https://github.com/smilebase/**logback-mongodb<https://github.com/smilebase/logback-mongodb>
[9]: https://github.com/uded/**logback-mongodb-appender<https://github.com/uded/logback-mongodb-appender>
On 14.06.2012 14:24, ceki wrote:
Hi Christian,
Tomasz created a jira issue for implementing MongoDBAppender [2]. His implementation can be found at [3, 4, 5].
I think his impl is worth a look. He seprated connection functionality into MongoDBAppenderBase (part of logback-core) and ILoggingEvent specific code into MongoDBAppender (in logback-classic). Moreover, his code comes with 100% test coverage!
I guess the code could be easily extended to cater for AccessEvent (in logback-access).
Anyway, let me know what I can do to move this work forward.
[1] https://github.com/qos-ch/**logback-extensions<https://github.com/qos-ch/logback-extensions> [2] http://jira.qos.ch/browse/**LOGBACK-522<http://jira.qos.ch/browse/LOGBACK-522> [3] https://github.com/nurkiewicz/**logback/tree/mongodb-appender<https://github.com/nurkiewicz/logback/tree/mongodb-appender> [4] https://github.com/nurkiewicz/**logback/commit/858aceb9af74d0<https://github.com/nurkiewicz/logback/commit/858aceb9af74d0> [5] https://github.com/nurkiewicz/**logback/commit/f5bff8e43<https://github.com/nurkiewicz/logback/commit/f5bff8e43>
pozdrowienia
-- Tomasz Nurkiewicz http://nurkiewicz.blogspot.com ______________________________**_________________ logback-dev mailing list logback-dev@qos.ch http://mailman.qos.ch/mailman/**listinfo/logback-dev<http://mailman.qos.ch/mailman/listinfo/logback-dev>
-- Christian Trutz Von-Flotow-Straße 24 D-45772 Marl Festnetz (privat): +49 (0)2365 3840327 E-Mail: christian.trutz@gmail.com
Sure, by definition *unit* tests should not depend on any external resource like databases. All my tests are /integration/ ones, testing the whole stack from logging to reading back from MongoDB. We should probably split them and write mocked tests as well (I like Mockito, but it is just a matter of preference). On 14.06.2012 19:54, Christian Trutz wrote:
Hi Tomasz,
The only issue I see is integration testing: at least my tests require running MongoDB instance somewhere. This is a major drawback for both continuous integration of Lobgack extensions and for all developers, not necessarily interested in this particular extension.
I think we should not require for unit test a mongodb instance. I have very good experience with jmockit http://code.google.com/p/jmockit/ what do you think about?
Christian
2012/6/14 Tomasz Nurkiewicz <nurkiewicz@gmail.com <mailto:nurkiewicz@gmail.com>>
Here is precisely what I created so far:
* base abstract appender handling MongoDB connectivity [1] * standard appender [2] + tests [3] * access appender (tested with Tomcat) [4] + tests [5] * configuration examples [6]
By quick googling I found several other community implementations: [7], [8], [9]. Looks like my implementation is the only one capable of storing access logs as well, e.g.:
{ "_id" : ObjectId("__4d98cc4f7abb95e59279e183"), "timeStamp" : ISODate("2011-04-03T19:36:47.__339Z"), "server" : "localhost", "remote" : { "host" : "0:0:0:0:0:0:0:1", "user" : "tomcat", "addr" : "0:0:0:0:0:0:0:1" }, "request" : { "uri" : "/manager/images/tomcat.gif", "protocol" : "HTTP/1.1", "method" : "GET", "sessionId" : "__1C6357816D9EEFD31F6D9D154D8730__8A", "userAgent" : "Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.2.16) Gecko/20110323 Ubuntu/10.10 (maverick) Firefox/3.6.16", "referer" : "http://localhost:8080/__manager/html <http://localhost:8080/manager/html>" }, "response" : { "contentLength" : NumberLong(1934), "statusCode" : 200 } }
We should now revise all available implementations and build final one based on best ideas from all of them. The only issue I see is integration testing: at least my tests require running MongoDB instance somewhere. This is a major drawback for both continuous integration of Lobgack extensions and for all developers, not necessarily interested in this particular extension.
Christian Trutz - thanks for creating the project structure.
[1]: https://github.com/nurkiewicz/__logback/blob/mongodb-appender/__logback-core... <https://github.com/nurkiewicz/logback/blob/mongodb-appender/logback-core/src/main/java/ch/qos/logback/core/db/mongo/MongoDBAppenderBase.java>
[2]: https://github.com/nurkiewicz/__logback/blob/mongodb-appender/__logback-clas... <https://github.com/nurkiewicz/logback/blob/mongodb-appender/logback-classic/src/main/java/ch/qos/logback/classic/db/mongo/MongoDBAppender.java>
[3]: https://github.com/nurkiewicz/__logback/blob/mongodb-appender/__logback-clas... <https://github.com/nurkiewicz/logback/blob/mongodb-appender/logback-classic/src/test/java/ch/qos/logback/classic/db/mongo/MongoDBAppenderTest.java>
[4]: https://github.com/nurkiewicz/__logback/blob/mongodb-appender/__logback-acce... <https://github.com/nurkiewicz/logback/blob/mongodb-appender/logback-access/src/main/java/ch/qos/logback/access/db/mongo/MongoDBAppender.java>
[5]: https://github.com/nurkiewicz/__logback/blob/mongodb-appender/__logback-acce... <https://github.com/nurkiewicz/logback/blob/mongodb-appender/logback-access/src/test/java/ch/qos/logback/access/db/mongo/MongoDBAppenderTest.java>
[6]: https://github.com/nurkiewicz/__logback/tree/mongodb-appender/__logback-clas... <https://github.com/nurkiewicz/logback/tree/mongodb-appender/logback-classic/src/test/input/joran/mongodb>
[7]: https://github.com/kofemann/__mongo-appender-for-logback <https://github.com/kofemann/mongo-appender-for-logback>
[8]: https://github.com/smilebase/__logback-mongodb <https://github.com/smilebase/logback-mongodb>
[9]: https://github.com/uded/__logback-mongodb-appender <https://github.com/uded/logback-mongodb-appender>
On 14.06.2012 14:24, ceki wrote:
Hi Christian,
Tomasz created a jira issue for implementing MongoDBAppender [2]. His implementation can be found at [3, 4, 5].
I think his impl is worth a look. He seprated connection functionality into MongoDBAppenderBase (part of logback-core) and ILoggingEvent specific code into MongoDBAppender (in logback-classic). Moreover, his code comes with 100% test coverage!
I guess the code could be easily extended to cater for AccessEvent (in logback-access).
Anyway, let me know what I can do to move this work forward.
[1] https://github.com/qos-ch/__logback-extensions <https://github.com/qos-ch/logback-extensions> [2] http://jira.qos.ch/browse/__LOGBACK-522 <http://jira.qos.ch/browse/LOGBACK-522> [3] https://github.com/nurkiewicz/__logback/tree/mongodb-appender <https://github.com/nurkiewicz/logback/tree/mongodb-appender> [4] https://github.com/nurkiewicz/__logback/commit/858aceb9af74d0 <https://github.com/nurkiewicz/logback/commit/858aceb9af74d0> [5] https://github.com/nurkiewicz/__logback/commit/f5bff8e43 <https://github.com/nurkiewicz/logback/commit/f5bff8e43>
pozdrowienia
-- Tomasz Nurkiewicz http://nurkiewicz.blogspot.com _________________________________________________ logback-dev mailing list logback-dev@qos.ch <mailto:logback-dev@qos.ch> http://mailman.qos.ch/mailman/__listinfo/logback-dev <http://mailman.qos.ch/mailman/listinfo/logback-dev>
-- Christian Trutz Von-Flotow-Straße 24 D-45772 Marl
Festnetz (privat): +49 (0)2365 3840327 E-Mail: christian.trutz@gmail.com <mailto:christian.trutz@gmail.com>
_______________________________________________ logback-dev mailing list logback-dev@qos.ch http://mailman.qos.ch/mailman/listinfo/logback-dev
pozdrowienia -- Tomasz Nurkiewicz http://nurkiewicz.blogspot.com
participants (3)
-
ceki -
Christian Trutz -
Tomasz Nurkiewicz