I also ended up with extending SiftingAppender:

    private static class ClosingSiftingAppender extends SiftingAppender {
        @Override
        protected void append(ILoggingEvent event) {
            super.append(event);
            if (eventMarksEndOfLife(event)) {
                getAppenderTracker().removeStaleComponents(Long.MAX_VALUE);
            }
        }
    }

Even if SiftingAppender would work (I suspect it doesn't because of a combination of timeout I've specified, frequency of the FINALIZE_SESSION markers sent, and actual append calls), it would not fit my needs as in our app we actually need to close the appender immediately, without any lingering or timeout.

I'd suggest to support this (immediate close) when timeout option is set to 0.

Another issue I've ran into is that JoranConfigurator uses its own ClassLoader to load classes, and as our code is a plugin loaded by a host app and JoranConfigurator is used by the host app and is already loaded, I couldn't specify a custom appender in the logback.xml and have to configure Loggers at runtime, programmatically.