Fullstop - Audit reporting

Fullstop AWS overview

Fullstop Architecture overview

Aim of the project is to enrich CloudTrail log events.

In our scenario we have multiple AWS accounts that need to be handled.

Each of this account has CloudTrail activated and is configured to write in a bucket that resides in the account where also fullstop is running. (Right now in AWS it's not possible to read CloudTrail logs from a different account)

Fullstop will then process events collected from CloudTrail.

To enrich CloudTrail log events with information that comes from other systems than AWS, we should only configure fullstop to do so.

Fullstop can even call the AWS API of a different account, by using a cross-account role. The account that is running fullstop should therefore be trusted by all other accounts in order to perform this operations.

Plugins

• fullstop-ami-plugin
• fullstop-application-lifecycle-plugin
• fullstop-hello-event-plugin
• fullstop-keypair-plugin
• fullstop-region-plugin
• fullstop-registry-plugin
• fullstop-save-security-group-plugin
• fullstop-scm-repository-plugin
• fullstop-snapshot-source-plugin
• fullstop-subnet-plugin
• fullstop-taupage-yaml-plugin
• fullstop-unapproved-services-and-role-plugin
• fullstop-lambda-plugin

Jobs

In addition to listening on cloudtrail events, Fullstop. runs jobs as well. A list of jobs can be found here.

Application lifecycle

Fullstop. also keeps record of your application lifecycles. for more information, see the [fullstop-application-lifecycle-plugin] (fullstop-plugins/fullstop-application-lifecycle-plugin) and the Application lifecycle API

Information sources

Fullstop uses different source to gather information.

Configuration

This environment variables should be set:

FULLSTOP_LOGS
FULLSTOP_SQS_URL
FULLSTOP_SQS_REGION
FULLSTOP_S3_REGION
FULLSTOP_TAUPAGE_OWNERS
FULLSTOP_TAUPAGE_NAME_PREFIX
FULLSTOP_S3_BUCKET
FULLSTOP_KIO_URL
FULLSTOP_PIERONE_URLS
FULLSTOP_TEAM_SERVICE_URL
DATABASE_URL
DATABASE_USER
DATABASE_PASSWORD
DATABASE_DRIVER
INSTANCE_LOGS_S3_BUCKET
ACCESS_TOKEN_URI
CREDENTIALS_DIR
TOKEN_INFO_URI
FULLSTOP_UNAPPROVED_SERVICES_AND_ROLE_BUCKET_NAME
FULLSTOP_UNAPPROVED_SERVICES_AND_ROLE_PREFIX

Example:

$ export FULLSTOP_LOGS=/fullstop_logs_dir
$ export FULLSTOP_SQS_URL=https://sqs.eu-central-1.amazonaws.com/ACCOUNT_ID/fullstop
$ export FULLSTOP_SQS_REGION=eu-central-1
$ export FULLSTOP_S3_REGION=eu-west-1
$ export FULLSTOP_TAUPAGE_OWNERS=999999999999
$ export FULLSTOP_TAUPAGE_NAME_PREFIX=Taupage
$ export FULLSTOP_S3_BUCKET=fullstop-bucket
$ export FULLSTOP_KIO_URL: https://application.registry.address
$ export FULLSTOP_PIERONE_URLS: https://docker.repository.address
$ export FULLSTOP_TEAM_SERVICE_URL: https://team.service.address
$ export DATABASE_URL='jdbc:postgresql://localhost:5432/fullstop'
$ export DATABASE_USER=postgres
$ export DATABASE_PASSWORD='{cipher}234laksnfdlF83NHALF'
$ export DATABASE_DRIVER=org.postgresql.Driver
$ export INSTANCE_LOGS_S3_BUCKET=my-s3-bucket
$ export ACCESS_TOKEN_URI=accessTokenUri
$ export CREDENTIALS_DIR=/location/credentials
$ export TOKEN_INFO_URI=tokenInfoUri
$ export FULLSTOP_UNAPPROVED_SERVICES_AND_ROLE_BUCKET_NAME=fullstop-bucket-policy
$ export FULLSTOP_UNAPPROVED_SERVICES_AND_ROLE_PREFIX=folder_containing_templates_files

Disable CloudTrail Processing

Set the parameter fullstop.container.autoStart=false either as program argument, or as system property to start Fullstop without CloudTrail processing.

Scopes

In order to create violations and save them to the database, your application which is responsible for that, needs this oAuth2 scope for the POST method on "/api/violations" endpoint

fullstop.violation.write

Database setup

Fullstop will store the violations in a RDBMS. Once you start Fullstop, it will create the necessary schema and tables for you. The database itself, however, has to be created by you. Your database password is encrypted with AWS KMS. We are using Taupage to decrypt the password on the fly. To use Amazons KMS for de/encryption, you need to to provide a region and the key id for your key. In Fullstop, both will be provided via environment variables.

The password should be already encrypted, when you store it in the DATABASE_PASSWORD environment variable. An encrypted password always starts with aws:kms:. You can use our CLI tool for encryption or you use Amazons AWS CLI.

Propose API changes

Important all changes should be swagger 2.0 spec compliant.

• Click here to edit the API definition.
• Copy your changes and paste it in the Github web editor: fullstop-api.yaml.
• Create a pull request with the new swagger 2.0 spec.

How to build

Needs Java 1.8. Newer jdks are not supported yet.

$ ./mvnw clean install

Run entire test suite (requires a local docker environment)

$ ./mvnw clean verify -Pintegration-test

How to run

You need a locally running PostgreSQL 9.4 server (database "fullstop", host "localhost", user "postgres", password "postgres").

$ # set env variables first
$ cd fullstop
$ mvn spring-boot:run -Dfullstop.container.autoStart=false

How to build a docker image

Build fullstop:

$ mvn clean package -U

Build scm-source.json:

$ ./scm-source.sh

Build docker image:

$ docker build -t registry/fullstop:0.1 fullstop

Show images:

$ docker images

Run with docker:

$ docker run -it registry/fullstop:0.1

Push docker image:

$ docker push registry/fullstop:0.1

License

Copyright © 2015 Zalando SE

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.