picturesafe-search

Service API for Elasticsearch integration

License

License

Categories

Categories

Search Business Logic Libraries
GroupId

GroupId

de.picturesafe.search
ArtifactId

ArtifactId

picturesafe-search
Last Version

Last Version

3.5.0
Release Date

Release Date

Type

Type

jar
Description

Description

picturesafe-search
Service API for Elasticsearch integration
Project URL

Project URL

https://github.com/picturesafe/picturesafe-search
Source Code Management

Source Code Management

https://github.com/picturesafe/picturesafe-search

Download picturesafe-search

How to add to project

<!-- https://jarcasting.com/artifacts/de.picturesafe.search/picturesafe-search/ -->
<dependency>
    <groupId>de.picturesafe.search</groupId>
    <artifactId>picturesafe-search</artifactId>
    <version>3.5.0</version>
</dependency>
// https://jarcasting.com/artifacts/de.picturesafe.search/picturesafe-search/
implementation 'de.picturesafe.search:picturesafe-search:3.5.0'
// https://jarcasting.com/artifacts/de.picturesafe.search/picturesafe-search/
implementation ("de.picturesafe.search:picturesafe-search:3.5.0")
'de.picturesafe.search:picturesafe-search:jar:3.5.0'
<dependency org="de.picturesafe.search" name="picturesafe-search" rev="3.5.0">
  <artifact name="picturesafe-search" type="jar" />
</dependency>
@Grapes(
@Grab(group='de.picturesafe.search', module='picturesafe-search', version='3.5.0')
)
libraryDependencies += "de.picturesafe.search" % "picturesafe-search" % "3.5.0"
[de.picturesafe.search/picturesafe-search "3.5.0"]

Dependencies

compile (8)

Group / Artifact Type Version
org.elasticsearch : elasticsearch jar 7.9.1
org.elasticsearch.client : elasticsearch-rest-high-level-client jar 7.9.1
org.elasticsearch.client : elasticsearch-rest-client-sniffer jar 7.9.1
org.apache.commons : commons-lang3 jar 3.11
commons-collections : commons-collections jar 3.2.2
commons-io : commons-io jar 2.7
org.slf4j : slf4j-api jar 1.7.30
com.fasterxml.jackson.core : jackson-databind Optional jar 2.11.2

provided (3)

Group / Artifact Type Version
org.springframework : spring-context jar 5.2.8.RELEASE
org.springframework : spring-beans jar 5.2.8.RELEASE
org.springframework : spring-tx jar 5.2.8.RELEASE

test (10)

Group / Artifact Type Version
org.apache.logging.log4j : log4j-core jar 2.13.3
org.apache.logging.log4j : log4j-slf4j-impl jar 2.13.3
org.elasticsearch.test : framework jar 7.9.1
junit : junit jar 4.13
org.mockito : mockito-core jar 1.10.19
org.springframework : spring-test jar 5.2.8.RELEASE
org.springframework : spring-aop jar 5.2.8.RELEASE
org.springframework : spring-jdbc jar 5.2.8.RELEASE
org.hsqldb : hsqldb jar 2.5.1
com.jayway.jsonpath : json-path jar 2.4.0

Project Modules

There are no modules declared in this project.

picturesafe-search

An Elasticsearch service wrapper

picturesafe-search is a Java service wrapper for the search engine Elasticsearch.

It allows the fast, flexible and easy integration of Elasticsearch functions into new or existing Java applications.

The following features are included:

  • Functions for creating and maintaining Elasticsearch indices

    • Create field definitions

    • Create and delete Elasticsearch indices

    • Add and remove documents

  • Query API for creating simple and complex (nested) search queries.

    • Fulltext queries

    • Field queries

    • Complex queries

  • Filter builder API for easy implementation of customized search filters

  • Aggregation builder API for easy implementation of customized aggregation builders (facets)

Make Elasticsearch queries easy

Elasticsearch is a very powerful search engine, but has a high complexity and requires a long learning curve.

The following example compares the execution of the logical search expression fulltext contains "(test && title)" and count >= 102 sort by id between the Elasticsearch REST API, the Elasticsearch Java High Level REST Client and the picturesafe-search Java service wrapper:

Elasticsearch REST API
POST /picturesafe-search-sample/_search
{
  "query": {
    "bool": {
      "must": [{
        "bool": {
          "filter": [{
            "query_string": {
              "query": "(test && title)",
              "fields": ["fulltext^1.0"]
            }
          }]
        }
      }],
      "filter": [{
        "range": {
          "count": {
            "from": 102,
            "to": null
          }
        }
      }]
    }
  },
  "sort": [{
    "id.keyword": {
      "order": "desc",
      "missing": "_last"
    }
  }]
}
Elasticsearch Java High Level REST Client
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder()
    .query(QueryBuilders.boolQuery()
    .must(QueryBuilders.queryStringQuery("test && title")
            .field("fulltext")
            .defaultOperator(Operator.AND))
    .filter(QueryBuilders.rangeQuery("count").gte(102)))
    .sort(SortBuilders
            .fieldSort("id.keyword")
            .missing("_last")
            .order(SortOrder.DESC));

SearchRequest searchRequest = new SearchRequest("picturesafe-search-sample");
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = new RestClientSearchAction()
    .action(restHighLevelClient, searchRequest);
picturesafe-search Java service wrapper
SearchResult searchResult = elasticsearchService.search("picturesafe-search-sample",
    OperationExpression.and(
        new FulltextExpression("test title"),
        new ValueExpression("count", ValueExpression.Comparison.GE, 102)),
    SearchParameter.builder().sortOptions(SortOption.desc("id")).build());

As you can see, picturesafe-search focuses more on WHAT to search for than on HOW to search it and abstracts some complexity.

Getting started

Only a few steps are necessary to get started with a standard configuration. A complete code example can be found here. If you want to create a Spring Boot application, you can alternatively use the picturesafe-search Starter.

For more information, please see following documentation.

Start Elasticsearch

picturesafe-search requires a running Elasticsearch server from version 7.x.

Local installation of Elasticsearch

For a new application an Elasticsearch server must be installed first:

  • Download and unpack the Elasticsearch official distribution.

  • Run bin/elasticsearch on Linux or macOS. Run bin\elasticsearch.bat on Windows.

Some features of picturesafe-search (for example sorting documents in a language-specific word order) require the ICU Analysis Plugin for Elasticsearch:

  • Run bin/elasticsearch-plugin install analysis-icu on Linux or macOS. Run bin\elasticsearch-plugin install analysis-icu on Windows.

Run Elasticsearch in a Docker container

As an alternative to installing Elasticsearch you can run it in a Docker container. To do this you can use the provided Docker Compose file.

  • Install Docker and Docker Compose.

  • Clone the picturesafe-search GitHub repository.

  • Run docker-compose -f src/test/docker/docker-compose.yml up -d from the project directory to start Elasticsearch.

  • To stop Elasticsearch run docker-compose -f src/test/docker/docker-compose.yml stop from the project directory.

Include java library

Add the current version of the picturesafe-search library to your project.

Maven dependency
<dependency>
    <groupId>de.picturesafe.search</groupId>
    <artifactId>picturesafe-search</artifactId>
    <version>3.6.0-SNAPSHOT</version>
</dependency>

Configuration

Configuration bean

Implement a configuration class that imports the DefaultElasticConfiguration.class. This configuration can be extended later.

The following example defines three fields for the Elasticsearch index:

  • Field 'id' (Elasticsearch type integer, sortable)

  • Field 'fulltext' (Elasticsearch type text)

  • Field 'title' (Elasticsearch type text, within fulltext, aggregatable, sortable)

Spring configuration
@Configuration
@ComponentScan(basePackages = {"de.picturesafe.search.elasticsearch"})
@Import({DefaultElasticConfiguration.class})
public class Config {

    @Bean
    List<FieldConfiguration> fieldConfigurations() {
        return Arrays.asList(
            FieldConfiguration.ID_FIELD,
            FieldConfiguration.FULLTEXT_FIELD,
            StandardFieldConfiguration.builder(
                    "title", ElasticsearchType.TEXT).copyToFulltext(true).sortable(true).build(),
            StandardFieldConfiguration.builder(
                    "count", ElasticsearchType.INTEGER).sortable(true).build()
        );
    }
}

Configuration properties

Add a file elasticsearch.properties to the classpath of your application and define the following key:

Property file
elasticsearch.index.alias=picturesafe-search-sample

This configuration can be extended later, see src/main/resources/elasticsearch.template.properties.

Service implementation

Inject the SingleIndexElasticsearchService and implement an expression-based search:

  • Create an Elasticsearch index with alias

  • Add some documents to the index

  • Create an OperationExpression with two terms

  • Run the search query

  • Delete the Elasticsearch index

If you want to implement searches for more than one index, please use ElasticsearchService instead of SingleIndexElasticsearchService.

Spring service implementation
@Component
@ComponentScan
public class GettingStarted {

    private static final Logger LOGGER = LoggerFactory.getLogger(GettingStarted.class);

    @Autowired
    private SingleIndexElasticsearchService singleIndexElasticsearchService;

    public static void main(String[] args) {
        try (AnnotationConfigApplicationContext ctx
                = new AnnotationConfigApplicationContext(GettingStarted.class)) {
            final GettingStarted gettingStarted = ctx.getBean(GettingStarted.class);
            gettingStarted.run();
        }
    }

    private void run() {
        try {
            singleIndexElasticsearchService.createIndexWithAlias();

            singleIndexElasticsearchService
                    .addToIndex(DataChangeProcessingMode.BLOCKING, Arrays.asList(
                    DocumentBuilder.id(1).put("title", "This is a test title")
                        .put("count", 101).build(),
                    DocumentBuilder.id(2).put("title", "This is another test title")
                        .put("count", 102).build(),
                    DocumentBuilder.id(3).put("title", "This is one more test title")
                        .put("count", 103).build()
            ));

            final Expression expression = OperationExpression.and(
                    new FulltextExpression("test title"),
                    new ValueExpression("count", ValueExpression.Comparison.GE, 102));

            final SearchResult searchResult = singleIndexElasticsearchService
                .search(expression, SearchParameter.DEFAULT);

            LOGGER.info(searchResult.toString());
        } finally {
            singleIndexElasticsearchService.deleteIndexWithAlias();
        }
    }
}

With implementations of the picturesafe-search Expression-Interface complex terms of different search conditions can be easily defined.

Here are some examples:

Simple fulltext search
Expression expression = new FulltextExpression("test title");
Simple field search
Expression expression = new ValueExpression("title", "test");
Simple field search with comparison operator
Expression expression = new ValueExpression("count", ValueExpression.Comparison.GE, 102);
Search with two terms
Expression expression = OperationExpression.and(
        new FulltextExpression("test title"),
        new ValueExpression("count", ValueExpression.Comparison.GE, 102));

In addition there are further expressions like InExpression, MustNotExpression, RangeValueExpression, DayExpression, more

If you want to build picturesafe-search yourself there are two prerequisites:

JDK

You need to have installed a Java Development Kit. The picturesafe-search project is currently developed using Java 8, but has also been tested on Java 11.

Note when using Java 11:
There is a JavaDoc related bug which has not been fixed in Adopt or Corretto OpenJDK at the moment. If you are using OpenJDK 11 and you are facing a build error like

Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:3.2.0:jar (attach-javadocs) on project picturesafe-search: MavenReportException: Error while generating Javadoc:

[ERROR] Exit code: 1 - javadoc: error - The code being documented uses packages in the unnamed module, but the packages defined in https://docs.oracle.com/en/java/javase/11/docs/api/ are in named modules.

, please skip generating JavaDoc until the fix has become part of the OpenJDK build you are using.

Skipping the JavaDoc generation:
mvn -Dmaven.javadoc.skip=true install

Alternatively you could use the OpenJDK 11 reference build provided by Oracle, which has the fix included.

Side note on java modules:
We are not able to provide a module-info.java at the moment, because we are using the Elasticsearch high level rest client which has the monolithic elasticsearch.jar as dependency. The elasticsearch.jar has no module-info and it makes auto module detection impossible because of its internal structure. Please see this issue for details.

Apache Maven

You also need to have installed Apache Maven version 3.6.

Build

Change to the project directory and run the following command in your shell:

mvn install
de.picturesafe.search

Apollon GmbH + Co. KG

Versions

Version
3.5.0
3.4.1
3.4.0
3.3.0
3.2.0
3.1.1
3.1.0