[logback-user] The utf-8 encode setting doesn't take effect if no unicode log at beginning of application

22 Apr 2016

      Hello,

I am using slf4j with logback as backend log solution. When I trying to make the log file support Unicode, I add <encoder><charset>UTF-8</charset><encoder> to logback.xml,
However I find problems.
My slf4j-api version 1.7.20
        Logback-classic and logback-core: 1.1.7

Code:
package cbao.cmdtests;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
* @author cbao
*
*/
public class LogTests {

    /**
     * @param args
     * @throws InterruptedException
     */
    public static void main(String[] args) throws InterruptedException {
        Logger logger = LoggerFactory.getLogger("LogTests");

        for (int i = 0; i < 5000; i++) {
            logger.info("This is ANSI text");
            Thread.sleep(0);
        }
        logger.info("This is unicode text: 中文");

    }
}

Logback.xml:
<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true">
    <appender name="consoleAppender" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <charset>UTF-8</charset>
            <Pattern>%d %-4relative [%thread] %-5level %logger{35} - %msg%n</Pattern>
        </encoder>
    </appender>

    <appender name="FILE" class="ch.qos.logback.core.FileAppender">
        <file>logs/application.log</file>
        <encoder>
            <charset>UTF-8</charset>
            <pattern>%d %-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
            <immediateFlush>true</immediateFlush>
        </encoder>
    </appender>

    <root level="DEBUG">
        <appender-ref ref="consoleAppender" />
        <appender-ref ref="FILE"/>
    </root>
</configuration>

After running above code, the log file format is 1252 (ANSI – Latin I)

The end of log file:
2016-04-21 17:00:12,666 151328 [main] INFO  LogTests - This is ANSI text
2016-04-21 17:00:12,717 151379 [main] INFO  LogTests - This is ANSI text
2016-04-21 17:00:12,767 151429 [main] INFO  LogTests - This is ANSI text
2016-04-21 17:00:12,817 151479 [main] INFO  LogTests - This is ANSI text
2016-04-21 17:00:12,867 151529 [main] INFO  LogTests - This is ANSI text
2016-04-21 17:00:12,918 151580 [main] INFO  LogTests - This is ANSI text
2016-04-21 17:00:12,968 151630 [main] INFO  LogTests - This is ANSI text
2016-04-21 17:00:13,019 151681 [main] INFO  LogTests - This is ANSI text
2016-04-21 17:00:13,070 151732 [main] INFO  LogTests - This is ANSI text
2016-04-21 17:00:13,120 151782 [main] INFO  LogTests - This is ANSI text
2016-04-21 17:00:13,171 151833 [main] INFO  LogTests - This is ANSI text
2016-04-21 17:00:13,222 151884 [main] INFO  LogTests - This is ANSI text
2016-04-21 17:00:13,273 151935 [main] INFO  LogTests - This is unicode text: ä¸æ–‡

As my test, if there is no unicode log, the newly created log file will be 1252 (ANSI – Latin I).
As soon as first unicode string logged, the log file changed (unicode: UTF-8).
However, if there are significant ansi text before first unicode text could be logged, the log file will stay as ANSI even after utf-8 logged, then the unicode log can’t be display correctly.

Is there any switches in logback.xml can fix it?

Thanks
Cheng Bao