JarCasting
JarCasting JarCasting

kafka-connect-transform-maxmind

Kafka Connect transformation for adding GeoIP data to Kafka data.

License

License
Categories

Categories

ORM Data
GroupId

GroupId

com.github.jcustenborder.kafka.connect
ArtifactId

ArtifactId

kafka-connect-transform-maxmind
Last Version

Last Version

0.1.0.7
Release Date

Release Date
Type

Type

tar.gz
Description

Description

kafka-connect-transform-maxmind
Kafka Connect transformation for adding GeoIP data to Kafka data.
Project URL

Project URL

https://github.com/jcustenborder/kafka-connect-transform-maxmind
Source Code Management

Source Code Management

https://github.com/jcustenborder/kafka-connect-transform-maxmind

Download kafka-connect-transform-maxmind

Filename Size
kafka-connect-transform-maxmind-0.1.0.7.pom
kafka-connect-transform-maxmind-0.1.0.7.tar.gz 6 MB
kafka-connect-transform-maxmind-0.1.0.7-sources.jar 10 KB
kafka-connect-transform-maxmind-0.1.0.7-javadoc.jar 25 KB
Browse

Dependencies

compile (5)

Group / Artifact Type Version
com.maxmind.geoip2 : geoip2 jar 2.9.0
com.fasterxml.jackson.core : jackson-databind jar 2.8.5
org.reflections : reflections jar 0.9.10
com.google.guava : guava jar 18.0
com.github.jcustenborder.kafka.connect : connect-utils jar [0.3.33,0.3.1000)

provided (1)

Group / Artifact Type Version
org.apache.kafka : connect-api jar 1.0.0

test (7)

Group / Artifact Type Version
com.github.jcustenborder.kafka.connect : connect-utils-testing-data jar [0.2.33,0.2.1000)
org.apache.commons : commons-compress jar 1.14
com.github.jcustenborder.kafka.connect : connect-utils-testing jar [0.3.33,0.3.1000)
org.junit.jupiter : junit-jupiter-engine jar 5.0.0
org.junit.jupiter : junit-jupiter-api jar 5.0.0
org.mockito : mockito-core jar 2.6.3
ch.qos.logback : logback-classic jar 1.1.8

Project Modules

There are no modules declared in this project.

Introduction

In order to build this locally you will need to download a copy of the GeoLite2 City from MaxMind.

Configuration

MaxMindTransformation

This transformation is used to lookup data from a MaxMind database and append the data to an existing struct.

transforms=maxmindtransformation
transforms.maxmindtransformation.type=com.github.jcustenborder.kafka.connect.transform.maxmind.MaxMindTransformation

# Set these required values
transforms.maxmindtransformation.field.input=
transforms.maxmindtransformation.maxmind.database.path=
transforms.maxmindtransformation.field.output=
Name Description Type Default Valid Values Importance
field.input The field in the value() of the connect record to read the ip address from. string high
field.output The field in the value() to write the GeoIp Data to. The schema will be appended. string high
maxmind.database.path The path of the database on the local file system. string high
maxmind.enable.anonymousip Flag to determine if the ip should be checked if it's used for anonymous internet access. boolean false medium
maxmind.enable.asn Flag to determine if the ip should be checked if ASN information should be included. boolean false medium
maxmind.enable.city Flag to determine if the ip should be checked if city information should be included. boolean true medium
maxmind.enable.connectiontype Flag to determine if the ip should be checked if connection information should be included. boolean false medium
maxmind.enable.country Flag to determine if the ip should be checked if country information should be included. boolean false medium
maxmind.enable.domain Flag to determine if the ip should be checked if domain information should be included. boolean false medium
maxmind.enable.enterprise Flag to determine if the ip should be checked if enterprise information should be included. boolean false medium
maxmind.enable.isp Flag to determine if the ip should be checked if ISP information should be included. boolean false medium
maxmind.database.file.mode The mode in which to open the database. string MEMORY_MAPPED ValidEnum{enum=FileMode, allowed=[MEMORY_MAPPED, MEMORY]} low

Schemas

com.maxmind.geoip2.model.GeoIpData

This schema contains all of the data that can be retrieved from the MaxMind api.

Name Optional Schema Default Value Documentation
anonymousIpResponse true com.maxmind.geoip2.model.AnonymousIpResponse
asnResponse true com.maxmind.geoip2.model.AsnResponse
cityResponse true com.maxmind.geoip2.model.CityResponse
connectionTypeResponse true com.maxmind.geoip2.model.ConnectionTypeResponse
countryResponse true com.maxmind.geoip2.model.CountryResponse
domainResponse true com.maxmind.geoip2.model.DomainResponse
enterpriseResponse true com.maxmind.geoip2.model.EnterpriseResponse
insightsResponse true com.maxmind.geoip2.model.InsightsResponse
ispResponse true com.maxmind.geoip2.model.IspResponse

com.maxmind.geoip2.record.RepresentedCountry

Name Optional Schema Default Value Documentation
type true String
name true String
names true Map of <String, String>
geoNameId true Int32

com.maxmind.geoip2.record.Continent

Name Optional Schema Default Value Documentation
code true String
name true String
names true Map of <String, String>
geoNameId true Int32

com.maxmind.geoip2.record.Location

Name Optional Schema Default Value Documentation
accuracyRadius true Int32
averageIncome true Int32
latitude true Float64
longitude true Float64
metroCode true Int32
populationDensity true Int32
timeZone true String

com.maxmind.geoip2.record.Postal

Name Optional Schema Default Value Documentation
code true String
confidence true Int32

com.maxmind.geoip2.record.City

Name Optional Schema Default Value Documentation
confidence true Int32
name true String
names true Map of <String, String>
geoNameId true Int32

com.maxmind.geoip2.record.Country

Name Optional Schema Default Value Documentation
confidence true Int32
isoCode true String
name true String
names true Map of <String, String>
geoNameId true Int32

com.maxmind.geoip2.record.Subdivision

Name Optional Schema Default Value Documentation
confidence true Int32
isoCode true String
name true String
names true Map of <String, String>
geoNameId true Int32

com.maxmind.geoip2.record.Traits

Name Optional Schema Default Value Documentation
autonomousSystemNumber true Int32
autonomousSystemOrganization true String
connectionType true String
domain true String
isAnonymousProxy true Boolean
isSatelliteProvider true Boolean
isp true String
organization true String
userType true String
isLegitimateProxy true Boolean

com.maxmind.geoip2.model.CountryResponse

Name Optional Schema Default Value Documentation
continent true com.maxmind.geoip2.record.Continent
country true com.maxmind.geoip2.record.Country
registeredCountry true com.maxmind.geoip2.record.Country
representedCountry true com.maxmind.geoip2.record.RepresentedCountry
traits true com.maxmind.geoip2.record.Traits

com.maxmind.geoip2.model.DomainResponse

Name Optional Schema Default Value Documentation
domain true String

com.maxmind.geoip2.model.ConnectionTypeResponse

Name Optional Schema Default Value Documentation
connectionType true String

com.maxmind.geoip2.model.CityResponse

Name Optional Schema Default Value Documentation
city true com.maxmind.geoip2.record.City
location true com.maxmind.geoip2.record.Location
postal true com.maxmind.geoip2.record.Postal
subdivisions true Array of com.maxmind.geoip2.record.Subdivision
continent true com.maxmind.geoip2.record.Continent
country true com.maxmind.geoip2.record.Country
registeredCountry true com.maxmind.geoip2.record.Country
representedCountry true com.maxmind.geoip2.record.RepresentedCountry
traits true com.maxmind.geoip2.record.Traits

com.maxmind.geoip2.model.AsnResponse

Name Optional Schema Default Value Documentation
autonomousSystemNumber true Int32
autonomousSystemOrganization true String

com.maxmind.geoip2.model.InsightsResponse

Name Optional Schema Default Value Documentation
city true com.maxmind.geoip2.record.City
location true com.maxmind.geoip2.record.Location
postal true com.maxmind.geoip2.record.Postal
subdivisions true Array of com.maxmind.geoip2.record.Subdivision
continent true com.maxmind.geoip2.record.Continent
country true com.maxmind.geoip2.record.Country
registeredCountry true com.maxmind.geoip2.record.Country
representedCountry true com.maxmind.geoip2.record.RepresentedCountry
traits true com.maxmind.geoip2.record.Traits

com.maxmind.geoip2.model.EnterpriseResponse

Name Optional Schema Default Value Documentation
city true com.maxmind.geoip2.record.City
location true com.maxmind.geoip2.record.Location
postal true com.maxmind.geoip2.record.Postal
subdivisions true Array of com.maxmind.geoip2.record.Subdivision
continent true com.maxmind.geoip2.record.Continent
country true com.maxmind.geoip2.record.Country
registeredCountry true com.maxmind.geoip2.record.Country
representedCountry true com.maxmind.geoip2.record.RepresentedCountry
traits true com.maxmind.geoip2.record.Traits

com.maxmind.geoip2.model.AnonymousIpResponse

Name Optional Schema Default Value Documentation
isAnonymous true Boolean
isAnonymousVpn true Boolean
isHostingProvider true Boolean
isPublicProxy true Boolean
isTorExitNode true Boolean

com.maxmind.geoip2.model.IspResponse

Name Optional Schema Default Value Documentation
isp true String
organization true String

Versions

Version
0.1.0.7
0.1.0.6
0.1.0.4
0.1.0.3