Project Nessie
Project Nessie is a system to provide Git like capability for Iceberg Tables, Delta Lake Tables, Hive Tables and Sql Views.
More information can be found at projectnessie.org.
Using Nessie
You can quickly get started with Nessie by using our small, fast docker image.
docker pull projectnessie/nessie
docker run -p 19120:19120 projectnessie/nessie
Then install the Nessie CLI tool
pip install pynessie
From there, you can use one of our technology integrations such those for
Have fun! We have a Google Group and a Slack channel we use for both developers and users. Check them out here.
Building and Developing Nessie
Requirements
- JDK 11 or higher: JDK11 or higher is needed to build Nessie (artifacts are built for with Java 8)
Installation
Clone this repository and run maven:
git clone https://github.com/projectnessie/nessie
cd nessie
./mvnw clean install
Delta Lake artifacts
Nessie required some minor changes to Delta for full support of branching and history. These changes are currently being integrated into the mainline repo. Until these have been merged we have provided custom builds in our fork which can be downloaded from a separate maven repository.
Distribution
To run:
- configuration in
servers/quarkus-server/src/main/resources/application.properties - execute
./mvnw quarkus:dev - go to
http://localhost:19120
UI
To run the ui (from ui directory):
- If you are running in test ensure that
setupProxy.jspoints to the correct api instance. This ensures we avoid CORS issues in testing npm installwill install dependenciesnpm run startto start the ui in development mode via node
To deploy the ui (from ui directory):
npm installwill install dependenciesnpm buildwill minify and collect the package for deployment inbuild- the
builddirectory can be deployed to any static hosting environment or run locally asserve -s build
Docker image
When running mvn clean install a docker image will be created at projectnessie/nessie which can be started with docker run -p 19120:19120 projectnessie/nessie and the relevant environment variables. Environment variables are specified as per https://github.com/eclipse/microprofile-config/blob/master/spec/src/main/asciidoc/configsources.asciidoc#default-configsources
AWS Lambda
You can also deploy to AWS lambda function by following the steps in servers/lambda/README.md