AisLib

Introduction

DMA AisLib is a Java library for handling AIS messages. This include

• Reading from AIS sources e.g. serial connection, TCP connection or file
• Handling of proprietary source tagging sentences
• Message filtering like doublet filtering and down sampling
• Decoding sentences and AIS messages
• Encoding sentences and AIS messages
• Sending AIS messages #6, #8, #12 and #14
• Handling application specific messages

The library contains test code and utility applications demonstrating the use.

Prerequisites

• Java 8
• Maven 3

Building

To build

mvn clean install

To run tests

mvn test

NOTE: Temporary manual build procedure described here: https://github.com/dma-ais/AisLib/issues/48#issuecomment-448634262.

Developing in Eclipse

Use M2 plugin or

mvn eclipse:eclipse

and import as regular project.

Contributing

You're encouraged to contribute to AisLib. Fork the code from https://github.com/dma-ais/AisLib and submit pull requests.

License

This library is licensed under the Apache License, Version 2.0.

Examples

Simple read and message handling

Reading an AisMessage from an sentence in code is easily done with the Vdm object whether it it is a single message or multiple.

For single messages (AisPositionMessage)

String aisSentence = "!AIVDM,1,1,,A,181:Jqh02c1Qra`E46I<@9n@059l,0*30";

Vdm vdm = new Vdm();
vdm.parse(aisSentence);

AisMessage aisMessage = AisMessage.getInstance(vdm);

For multiple messages (AisMessage5)

String aisSentence1 = "!AIVDM,2,1,9,B,53nFBv01SJ<thHp6220H4heHTf2222222222221?50:454o<`9QSlUDp,0*09"; 
String aisSentence2 = "!AIVDM,2,2,9,B,888888888888880,2*2E";

Vdm vdm = new Vdm();
vdm.parse(aisSentence1);
vdm.parse(aisSentence2);

AisMessage aisMessage = AisMessage.getInstance(vdm);

Reading from files or TCP/UDP connections is very simple with AisLib. In the example below messages are read from a file.

AisReader reader = AisReaders.createReaderFromInputStream(new FileInputStream("sentences.txt"));
reader.registerHandler(new Consumer<AisMessage>() {
	@Override
	public void accept(AisMessage aisMessage) {
		System.out.println("message id: " + aisMessage.getMsgId());
	}
});
reader.start();
reader.join();

Reading using a TCP connection:

AisReader reader = AisReaders.createReader("localhost", 4001);
reader.registerHandler(new Consumer<AisMessage>() {
	@Override
	public void accept(AisMessage aisMessage) {
		System.out.println("message id: " + aisMessage.getMsgId());
	}
});
reader.start();
reader.join();

If the connection is broken the reader will try to reconnect after a certain amount of time that can be set with:

reader.setReconnectInterval(1000);

A read timeout can be defined for the reader. If no data is received within this period the connection will be closed and a reconnect will be tried.

Reading using an UDP connection:

AisReader reader = AisReaders.createUdpReader(8888);
reader.registerHandler(new Consumer<AisMessage>() {
	@Override
	public void accept(AisMessage aisMessage) {
		System.out.println("message id: " + aisMessage.getMsgId());
	}
});
reader.start();
reader.join();

Reading raw message packets

Instead of working with AisMessage objects, it is possible to work with unparsed raw message packets (proprietary tags, comment blocks and VDM's). A packet consumer is registred in a reader.

AisReader reader = AisReaders.createReader("localhost", 4001);
reader.registerPacketHandler(new Consumer<AisPacket>() {
	@Override
	public void accept(AisPacket packet) {
		try {
			AisMessage message = packet.getAisMessage();
		} catch (AisMessageException | SixbitException e) {
			// Handle
            		return;
		}
		// Alternative returning null if no valid AIS message
		AisMessage message = packet.tryGetAisMessage();

		Date timestamp = packet.getTimestamp();
		CommentBlock cb = packet.getVdm().getCommentBlock();
	}
});
reader.start();
reader.join();

Alternatively packets can be read as a packet stream.

Path path = ...
try (AisPacketReader pReader = AisPacketReader.createFromFile(path, true)) {
	AisPacket p;
	while ((p = pReader.readPacket()) != null) {
		count++;
		// Handle packet
		...
	}
}

If the filename has '.zip' suffix decompression will automatically be applied.

Reader factory methods

An AisReader instance is created using factory methods in AisReaders. The following methods can be used.

• createReader(String hostname, int port) - Creates a reader connection to host:port.
• createReader(String commaHostPort) - Creates a {@link AisTcpReader} from a list of one or more hosts on the form: host1:port1,...,hostN:portN. If more than one host/port round robin will be used.
• createUdpReader(int port) - Creates a UDP reader reading from port on any interface.
• createUdpReader(String address, int port) - Creates a UDP reader reading from port on interface with address.
• createReaderFromInputStream(InputStream inputStream) - Creates a reader reading from an input stream.
• createReaderFromFile(String filename) - Creates a reader reading from a file. If the filename suffix is '.zip' or '.gz', zip or gzip decompression will be applied respectively.

Reader group

A collection of readers can be organized in an AisReaderGroup that will deliver packets in a single join packet stream. Sources are defined by a commaseparated list of host:port. If more than one host for each source, round robin reading is used.

List<String> sources = ...
sources.add("s1host1:s1port1,s1host2:s1port2,s1host3:s1port3);
sources.add("s2host1:s1port1,s2host2:s2port2);
...
AisReaderGroup g = AisReaders.createGroup("name", sources);

TBD

Working with messages

To determine what messages are received the instanceof operator can be used. The example below shows how to test and cast, and take advantage of the object oriented design where related messages shares parents.

@Override
public void accept(AisMessage aisMessage) {
	// Handle AtoN message
	if (aisMessage instanceof AisMessage21) {
		AisMessage21 msg21 = (AisMessage21) aisMessage;
		System.out.println("AtoN name: " + msg21.getName());
	}
	// Handle position messages 1,2 and 3 (class A) by using their shared parent
	if (aisMessage instanceof AisPositionMessage) {
		AisPositionMessage posMessage = (AisPositionMessage)aisMessage;
		System.out.println("speed over ground: " + posMessage.getSog());
	}
	// Handle position messages 1,2,3 and 18 (class A and B)
	if (aisMessage instanceof IGeneralPositionMessage) {
		IGeneralPositionMessage posMessage = (IGeneralPositionMessage)aisMessage;
		System.out.println("course over ground: " + posMessage.getCog());
	}
	// Handle static reports for both class A and B vessels (msg 5 + 24)
	if (aisMessage instanceof AisStaticCommon) {
		AisStaticCommon staticMessage = (AisStaticCommon)aisMessage;
		System.out.println("vessel name: " + staticMessage.getName());
	}
}

See UML diagram of messages

Outputting Human Readable JSON

AisMessages can be decoded to human readable JSON messages automatically with a single handler.

String aisSentence = "!AIVDM,1,1,,A,181:Jqh02c1Qra`E46I<@9n@059l,0*30";
Decoder decoder = new Decoder(aisSentence);
String json = decoder.decode(true);

outputs

{
  "msgId": 1,
  "repeatDFO": {
    "ais_value": 0,
    "decoded_text": "Message has been repeated 0 times"
  },
  "userId": 0,
  "navStatusDFO": {
    "ais_value": 0,
    "decoded_text": "Under way using engine"
  },
  "rotDFO": {
    "ais_value": 0,
    "decoded_text": "Turning right at 0.0 degrees/ min"
  },
  "sogDFO": {
    "ais_value": 171,
    "decoded_text": "17.1 knots"
  },
  "posAccDFO": {
    "ais_value": 0,
    "decoded_text": "Low (> 10 m) (default)"
  },
  "position": {
    "latitude": 36.8121133,
    "longitude": 21.3901667
  },
  "cogDFO": {
    "ais_value": 3136,
    "decoded_text": "313.6 degrees"
  },
  "trueHeadingDFO": {
    "ais_value": 315,
    "decoded_text": "315 degrees"
  },
  "utcSecDFO": {
    "ais_value": 8,
    "decoded_text": "8"
  },
  "specialManIndicatorDFO": {
    "ais_value": 0,
    "decoded_text": "Not available"
  },
  "raimDFO": {
    "ais_value": 0,
    "decoded_text": "Raim not in use"
  },
  "syncStateDFO": {
    "ais_value": 0,
    "decoded_text": "Utc direct"
  }
}

With the decoder you can also pass in any AisMessage data type for the same result

Multiple sources

The example below shows how messages from multiple sources can be handled by a single handler.

// Make a handler
Consumer<AisMessage> handler = new Consumer<AisMessage>() {
	@Override
	public void accept(AisMessage aisMessage) {
		System.out.println("aisMessage: " + aisMessage);
	}
};

// Make readers and register handler
AisReader reader1 = AisReaders.createReader("host1", 4001);
AisReader reader2 = AisReaders.createReader("host2", 4001);
AisReader reader3 = AisReaders.createReader("host3", 4001);
reader1.registerHandler(handler);
reader2.registerHandler(handler);
reader3.registerHandler(handler);

// Start readers
reader1.start(); reader2.start(); reader3.start();

Alternatively an AisReaderGroup can be used.

Round robin reading

The example below shows how to round robin between TCP hosts. If one goes down, the re-connect will be to the next on the list.

AisReader reader = AisReaders.createReader("host1:port1,host2:port2,host3:port3");
reader.registerHandler(new Consumer<AisMessage>() {
	@Override
	public void accept(AisMessage aisMessage) {
		System.out.println("message id: " + aisMessage.getMsgId());
	}
});
reader.start();
reader.join();

Message filtering

Message filters implement a single method

 boolean rejectedByFilter(AisMessage message);

Down sample example:

AisReader reader = AisReaders.createReader("ais163.sealan.dk:4712");
reader.registerHandler(new Consumer<AisMessage>() {
	DownSampleFilter downSampleFilter = new DownSampleFilter(60);
	@Override
	public void accept(AisMessage aisMessage) {
		if (downSampleFilter.rejectedFilter(aisMessage)) {
			return;
		}
		// Handle message
	}
});
reader.start();
reader.join();

A MessageHandlerFilter can be used to put in between readers and handlers. In the example below two filters are used. A doublet filter and a down sampling filter.

Consumer<AisMessage> handler = new Consumer<AisMessage>() {
	@Override
	public void accept(AisMessage aisMessage) {
		// Handle doublet filtered down sampled messages
	}
};

// Make down sampling filter with sampling rate 1 min and
// make handler the recipient of down sampled messages
MessageHandlerFilter downsampleFilter = new MessageHandlerFilter(new DownSampleFilter());
downsampleFilter.registerReceiver(handler);

// Make doublet filter with default window of 10 secs.
// Set down sample filter as recipient of doublet filered messages
MessageHandlerFilter doubletFilter = new MessageHandlerFilter(new DuplicateFilter());
doubletFilter.registerReceiver(downsampleFilter);

// Make reader
AisReader reader = AisReaders.createReader(host, port);
reader.registerHandler(doubletFilter);
reader.start();

Expression based packet filtering

It is also possible to perform packet filtering based on packet sources and contents using free-text expressions. These expressions must comply with a specified grammar.

Using the filter

The expression filter can be used programmatically like this:

Filtering on packets source

import static dk.dma.ais.packet.AisPacketFiltersExpressionFilterParser.parseExpressionFilter;
...
parseExpressionFilter("s.country = DNK & s.type = LIVE").test(aisPacket);

In this example the test method wll return true for all packets received from a source in Denmark ('DNK') and coming from a terrestrial source (typically a VHF base station as opposed to e.g. data received via satellite). For packets coming sources not fulfilling this expression the test method will return false.

Filtering on source attributes is indicated by the 's' in front of the field ('country').

Filtering on message contents

import static dk.dma.ais.packet.AisPacketFiltersExpressionFilterParser.parseExpressionFilter;
...
parseExpressionFilter("m.pos witin circle(37.9121, -122.4229, 2000)").test(aisPacket);

In this example the test method will return false for packets containing a position outside a cartesian circle centered in 37.912 degrees north (37°N54'44"), 122.4229 degrees west (22°W25'22"), and with a radius of 2000 meters. For other packets (including packets without any position information) the method will return true.

Filtering on message (~packet) attributes is indicated by the 'm.' in front of the field ('pos').

Filtering on targets

In some cases it is insufficient to filter on packet sources and content alone. For instance it may be desirable to block AIS packets with static and voyage related data (see class AisStaticCommon) for vessels which are outside a given area. Since the AisStaticCommon messages do not contain any position information, it is not possible to filter on these packets alone.

Instead, it is possible to use an expression filter which will "remember" all AIS packets it has previously been served. It uses these messages to track all AIS targets and keep a cache of the latest known positions, speeds, and other characteristis. This why, it is possible to filter on the target's characteristics rather than the latest packet received.

As an example:

import static dk.dma.ais.packet.AisPacketFiltersExpressionFilterParser.parseExpressionFilter;
...
parseExpresionFiter("t.sog > 10").test(aisPacket);

Of all packets send to the test-method only those which are related to a vesel with a speed over ground larger than 10 knots will result in true being returned from test. This is true even for e.g. AIS messages of type 5 which contain no speed information.

Filtering on target attributes is indicated by the 't.' in front of the field ('sog').

Example application

An example of an existing application which uses the expression filter is AisFilter which is based on main class dk.dma.ais.utils.filter.AisFilter:

java dk.dma.ais.utils.filter.Aisilter -t localhost:4001 -exp ".sog in 2..8"

This will make the application connect via TCP to localhost:4001 to receive AIS packets, filter them using the supplied expression (so that packets with speed over ground outside the range 2 to 8 knots are rejected), and output the remaining packets on the standard output.

Grammar

The full grammar is specified usit ANTLR notation in file expresionFilter.g4.

The following are examples of filter expressions:

Simple comparisons

• m.sog = 0
• m.sog != 0
• m.sog > 6.0
• m.sog < 7.0
• m.sog >= 6.6
• m.sog <= 6.6

In lists

• m.month = jan,feb,mar

• m.month = (jan,feb,mar)

• m.month in jan,feb,mar

• m.month in (jan,feb,mar)

• m.month @ jan,feb,mar

• m.month @ (jan,feb,mar)

• m.mmsi in 220431000,220435325,220430999

• m.mmsi in (220431000,220435325,220430999)

• m.month != jan,feb,mar

• m.month != (jan,feb,mar)

• m.month not in jan,feb,mar

• m.month not in (jan,feb,mar)

• m.month !@ jan,feb,mar

• m.month !@ (jan,feb,mar)

• m.mmsi not in 220431000,220435325,220430999

• m.mmsi not in (220431000,220435325,220430999)

In ranges

• m.id = 5..15

• m.id in 5..15

• m.id in (5..15)

• m.id @ (5..15)

• m.id != 5..15

• m.id not in 5..15

• m.id !@ (5..15)

Within geographical areas

• m.pos within circle(37.9058, -122.4310, 1000)
• m.pos within bbox(37.9058, -122.4310, 37.9185, -122.4149)

With string resembling glob

• m.name like D*
• m.name LIKE DI*
• m.name LIKE DI?NA
• m.name ~ D*A
• m.name ~ 'DIA*'

Named values

• m.type = TANKER
• m.type = military
• m.type in MILITARY, TANKER, HSC, FISHING
• m.type in 'tanker', 'military', HSC

Composite expressions with boolean and/or

• m.id = 1 & m.sog >= 6.1 & m.sog <= 6.9
• m.sog > 6.6 | m.sog < 6.4
• m.sog > 6.6 & m.sog < 6.7
• s.type=SAT|s.id=AISD|s.id=MSSIS

Fields

The following fields can be used in filter expressions

       | s.bs      | Source base station        |
       | s.country | Source country             | DNK
       | s.type    | Source type                | LIVE, SAT 
       | s.region  | Source region              | 0

(*) pos is represents same values as (lat, lon) but is used in contexts where complete position (not just latitude or longitude) is used.

Packet transformers

TBD. See dk.dma.ais.transform.*, dk.dma.ais.transform.TransformTest and dk.dma.ais.packet.AisPacketTagsTest.

Reading proprietary tags

AisLib can handle some proprietary tags inserted before VDM sentences, but implementations of factories must be given. In the example below Gatehouse source tags are handled.

Proprietary factories are defined in the file

ais-lib-messages/src/main/resources/META-INF/services/dk.dma.ais.proprietary.ProprietaryFactory

AisReader reader = AisReaders.createReaderFromInputStream(new FileInputStream("sentences.txt"));
reader.registerHandler(new Consumer<AisMessage>() {
	@Override
	public void accept(AisMessage aisMessage) {
		if (aisMessage.getSourceTag() != null) {
			IProprietarySourceTag sourceTag = aisMessage.getSourceTag();
			System.out.println("timestamp: " + sourceTag.getTimestamp());
		}
	}
});
reader.start();
reader.join();

AIS metadata

Besides from propritary tags, comment blocks carry metadata about the VDM sentences carrying the AIS message.

The example below shows how to handle comment blocks.

AisReader reader = ...
reader.registerPacketHandler(new Consumer<AisPacket>() {
    @Override
    public void accept(AisPacket aisPacket) {
    	Vdm vdm = packet.getVdm();
        if (vdm == null) {
            return;
        }
        CommentBlock cb = vdm.getCommentBlock();
        if (cb != null) {
        	String source = cb.getString();
        	Long cbTimestamp = cb.getTimestamp();
        }
    }
});
reader.start();
reader.join();

Common metadata has been standardized in the class AisPacketTags. The following attributes are used:

• Timestamp
• Source id
• Source basestation
• Source country (ISO three letter)
• Source type SAT | LIVE

Example

AisPacketTags tags = packet.getTags();
Assert.assertEquals(tags.getTimestamp().getTime(), 1354719387000L);
Assert.assertEquals(tags.getSourceId(), "SISSM");
Assert.assertEquals(tags.getSourceBs().intValue(), 2190047);
Assert.assertEquals(tags.getSourceCountry().getThreeLetter(), "DNK");

Full timestamp of VDM sentences are done in different proprietary fashions. AisLib tries to get the timestamp in three ways

1. Comment block key 'c'
2. Gatehouse propritary source tag
3. MSSIS timestamp appended to VDM sentence (the first occurence of a timestamp is used)

The timestamp is retrived from a packet using the following

Packet packet = ...
Date timestamp = packet.getTimestamp();

AisBus

TBD. See dk.dma.ais.bus.AisBusTest.

Sending a message

Example of sending an addressed text message in an ABM sentence. See test cases on how to send application specific message. See AisReader for different sending options. In the example below all ABM packaging is handled by AisReader.

AisReader aisReader = new AisTcpReader(host, port);
aisReader.start();
...
// Make AIS message 12
AisMessage12 msg12 = new AisMessage12();
msg12.setDestination(destination);
msg12.setMessage(message);

// Send using a blocking call
Abk abk = aisReader.send(msg12, 1, destination);
if (abk.isSuccess()) {
	...
} else {
	...
}