I copied the JoranCofigurator and replaced Pattern with ElementSelector and the original issue seems to get resolved.
However, the audit method calls the AuditorFactory to get the Auditor that I thought I was setting in the configure resource call.
Instead the auditor is null because the default auditor is returned, but the default auditor is not updated to point to the auditor I passed in.

AuditorFactory.configureByResource(auditor, Loader.getResource("logback-audit.xml", getClass().getClassLoader()));
new AuditorFacade(message.getClass().getSimpleName(), status.getName(), message.toString()).audit();

public void audit() throws AuditException {
    Auditor auditor = AuditorFactory.getAuditor();

public static Auditor getAuditor() {
    return defaultAuditor;
  }