kafka-logback

Configurable kafka appender for logback

License

License

Categories

Categories

SBE Data Data Structures Logback Application Layer Libs Logging
GroupId

GroupId

ru.sberned
ArtifactId

ArtifactId

kafka-logback
Last Version

Last Version

1.0.2
Release Date

Release Date

Type

Type

jar
Description

Description

kafka-logback
Configurable kafka appender for logback
Project URL

Project URL

https://github.com/Sberned/kafka-logback/
Source Code Management

Source Code Management

https://github.com/Sberned/kafka-logback

Download kafka-logback

How to add to project

<!-- https://jarcasting.com/artifacts/ru.sberned/kafka-logback/ -->
<dependency>
    <groupId>ru.sberned</groupId>
    <artifactId>kafka-logback</artifactId>
    <version>1.0.2</version>
</dependency>
// https://jarcasting.com/artifacts/ru.sberned/kafka-logback/
implementation 'ru.sberned:kafka-logback:1.0.2'
// https://jarcasting.com/artifacts/ru.sberned/kafka-logback/
implementation ("ru.sberned:kafka-logback:1.0.2")
'ru.sberned:kafka-logback:jar:1.0.2'
<dependency org="ru.sberned" name="kafka-logback" rev="1.0.2">
  <artifact name="kafka-logback" type="jar" />
</dependency>
@Grapes(
@Grab(group='ru.sberned', module='kafka-logback', version='1.0.2')
)
libraryDependencies += "ru.sberned" % "kafka-logback" % "1.0.2"
[ru.sberned/kafka-logback "1.0.2"]

Dependencies

compile (6)

Group / Artifact Type Version
ch.qos.logback : logback-classic jar 1.2.1
ch.qos.logback.contrib : logback-jackson jar 0.1.5
ch.qos.logback.contrib : logback-json-classic jar 0.1.5
com.fasterxml.jackson.core : jackson-databind jar 2.8.7
org.apache.kafka : kafka-clients jar 0.10.1.0
org.assertj : assertj-core jar 3.6.1

provided (1)

Group / Artifact Type Version
org.projectlombok : lombok jar 1.16.12

test (2)

Group / Artifact Type Version
junit : junit jar 4.12
org.mockito : mockito-core jar 2.2.26

Project Modules

There are no modules declared in this project.

Kafka logback appender

This is a logback appender for Kafka (0.10.1.0 kafka-client used) This appender expects topic, bootstrapServers, valueSerializer and layout as mandatory. You could also supply additional kafka customProps in <customProps> tag in the following manner <customProps>key1|value1,key2|value2</customProps>

We recommend wrapping this appender inside AsyncAppender in order not to block your application (consider setting neverBlock to true). This appender ships with CustomJsonLayout, which extends JsonLayout from logback. It inherits all its properties and adds its own in the same manner. List of properties added: includeLineNumber, includeClassName, includeMethodName, includeHost, includeFileName all set to true by default. You can also add any constant fields via additionalFields (see sample config below).

Do not forget to add `true` to AsyncAppender in case you need caller data (method, class, file, line number). Kafka appender could be used with any other layout rather than Json, just think about proper kafka value Serializer.

Sample configuration

<?xml version="1.0" encoding="UTF-8"?>
<configuration>

    <appender name="KAFKA" class="ru.sberned.kafkalogback.KafkaAppender">
        <topic><your_topic_name></topic>
        <bootstrapServers><server1>,<server2></bootstrapServers>
        <valueSerializer>org.apache.kafka.common.serialization.StringSerializer</valueSerializer>
        <failOnStartup>false</failOnStartup>
        <customProp>acks|all</customProp>
        <layout class="ru.sberned.kafkalogback.CustomJsonLayout">
            <timestampFormat>yyyy-MM-dd'T'HH:mm:ssZ</timestampFormat>
            <jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter"/>
            <additionalField>environment|${KAFKA_ENVIRONMENT:-dev}</additionalField>
        </layout>
    </appender>

    <appender name="ASYNC" class="ch.qos.logback.classic.AsyncAppender">
        <neverBlock>true</neverBlock>
        <includeCallerData>true</includeCallerData>
        <appender-ref ref="KAFKA" />
    </appender>

    <!-- Do not sent kafka client logs to kafka. This way you could fill all the queue of AsyncAppender-->
    <logger name="org.apache.kafka" level="INFO" additivity="false">
        <appender-ref ref="CONSOLE" />
        <appender-ref ref="FILE" />
    </logger>

    <!-- Please note. CONSOLE and FILE appenders are not defined in this example -->
    <root level="TRACE">
        <appender-ref ref="CONSOLE" />
        <appender-ref ref="FILE" />
        <appender-ref ref="ASYNC" />
    </root>

</configuration>

Versions

Version
1.0.2