Spring boot and Lucene configuration example

In this tutorial, we will set up a Spring boot application to use Hibernate search with a Lucene indexing backend. This tutorial assumes that you already have a working Spring boot application with JPA/Hibernate configured. Please check out our tutorial “Getting started with Spring boot 2 and JPA” if you are new to Spring boot and Hibernate.

I would also recommend that you read our article “Configuring and mapping database tables to JPA entities” if you are not familiar with mapping database tables to JPA entities.

Introduction

Hibernate search is an opensource library that integrates easily with existing Hibernate ORM/JPA systems. When Hibernate Search is installed onto an application, it performs two functions.

First, it provides an indexing API to be used for your indexing configuration. For example, you may decide to index the bank account numbers in your banking application, as it is an often searched term. For this, you can use certain annotations from Hibernate Search to mark that field for indexing.

The second function that Hibernate Search performs is to integrate your application with other data indexing, search and analysis systems. Such technologies include Apache Lucene, Elastic Search and Solr. Hibernate Search will integrate the indexed data with these systems. In other words, only the data that you marked for inclusion in the index will be available on these systems.

Apache Lucene is an opensource indexing and text search library. Lucene manages to do these tasks very efficiently, causing it to become not just popular, but also as the basic building block of numerous other systems, such as Elastic search, Apache Solr and many more.

In this tutorial, we will focus on configuring Hibernate Search with Lucene as the search technology onto a Spring boot application.

Adding the required dependencies

In order to get started with the configuration, we will need to add Hibernate search dependency in our application. We will add the following dependency in to our pom.xml file.

		<dependency>
			<groupId>org.hibernate</groupId>
			<artifactId>hibernate-search-orm</artifactId>
			<version>5.11.1.Final</version>
		</dependency>

As of the time of the writing of this article, the most recent version of the hibernate-search-orm dependency is 5.11.1.Final. Please make sure to check the maven repository here in order find out the latest version to add to your application.

Configuring the index location

Lucene provides indexing capabilities to other systems that utilize it’s API, in this case, Hibernate Search. When indexing data, the resulting indices can be stored locally in the filesystem, on a remote system such as Elastic Search or Solr, or on the cloud.

In this tutorial, we will store the indices  on the local filesystem for the sake of simplicity. Let us edit our application.properties file by adding the following properties.

spring.jpa.properties.hibernate.search.default.directory_provider=filesystem
spring.jpa.properties.hibernate.search.default.indexBase=./data/lucene

The “spring.jpa.properties.hibernate.search.default.directory_provider” property indicates the location where the index files will be written. This is indicated by the usage of the “filesystem” value. The “spring.jpa.properties.hibernate.search.default.indexBase” indicates the location where the lucene index files will be written.

Please note that the spring.jpa.properites. suffix is added for Spring-Boot’s autoconfiguration to pick up the properties and configure Hibernate Search for us. If you are not using Spring-boot, then you can omit this suffix, but you will need to add these properties to a “Hibernate.properties” file or configure these properties programmatically.

Configuring indices on JPA entities 

Now that we have all the required configurations in place, it is time to configure the JPA entities for indexing. Let us take the following simple entity as an example.

package com.nullbeans.persistence.models;

import javax.persistence.Entity;
import javax.persistence.Id;

@Entity
public class Person {
    
    private long id;

    private String name;

    @Id
    public long getId() {
        return id;
    }

    public void setId(long id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }
}

The @Indexed annotation

In order to mark an entity for indexing, we will need to add the @Indexed annotation to the entity. This tells Hibernate Search that the entity contains indexed fields. This in turn causes Lucene to create an index file for the entity.

package com.nullbeans.persistence.models;

import org.hibernate.search.annotations.Indexed;

import javax.persistence.Entity;
import javax.persistence.Id;

@Entity
@Indexed
public class Person {

Please note that if you add the @Indexed annotation to a JPA entity class, the entity’s Id field will automatically be added to the index. This means that Id fields will not need any extra configuration to be included to the index. However, you can still add extra annotations to the Id field if you would like to change the default indexing behavior for that field.

The @Field annotation

In order to include an entity field into the index file, we will need to add the @Field annotation to the field. For example, to mark the “name” field for indexing, we will configure the field’s getter method as follows.

    @Field
    public String getName() {
        return name;
    }

Configuring Spring-boot to create the index files

There is one extra step that is required in order to complete our Lucene configuration, and that is to create the index files. By default, Hibernate search will not create the files unless explicitly instructed to do so.

This can be done by using the FullTextEntityManager. If you have an efficient database design, then recreating the full index should take only a few seconds. Therefore, we recommend that you trigger the reindexing on application startup. There are few reasons for this:

• If your database was modified while your Spring-boot application is down, for example, due to maintenance or upgrades.
• If your application did not shutdown properly, there is no guarantee that your database and your index will be in sync. Therefore its better to trigger the indexing in order to have both the database and the index consistent.
• To clean up any junk modifications that may have occurred in the index files while the application was down.

In order to achieve this behavior, we will create a new Spring bean called “LuceneIndexSupport”. On application startup, this bean will be created and through this bean, we will trigger the indexing. Let us start by first creating the bean.

package com.nullbeans.accounting.services;

import org.hibernate.search.jpa.FullTextEntityManager;
import org.hibernate.search.jpa.Search;

import javax.persistence.EntityManagerFactory;

public class LuceneIndexServiceBean {

    private FullTextEntityManager fullTextEntityManager;

    public LuceneIndexServiceBean(EntityManagerFactory entityManagerFactory){
        fullTextEntityManager = Search.getFullTextEntityManager(entityManagerFactory.createEntityManager());
    }

    public void triggerIndexing() {
        try {
            fullTextEntityManager.createIndexer().startAndWait();
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        }
    }
}

Having the FullTextEntityManager contained in a bean has the advantage that it lives within your application context. Hence, it can be reused later on in the application to manually (or automatically) cleanup and trigger the indexing.

The call fullTextEntityManager.createIndexer().startAndWait() is a synchronous call. Therefore, it is recommended to be used only if you have an efficient Lucene index design. If the operation takes a long time, then we recommend that you explore other options, such as the .start() asynchronus call.

Now, let us start up this bean in a Java configuration file.

package com.nullbeans.accounting.config;

import com.nullbeans.accounting.services.LuceneIndexServiceBean;
import org.springframework.context.annotation.Bean;

import javax.persistence.EntityManagerFactory;

public class LuceneIndexConfig {

    @Bean
    public LuceneIndexServiceBean luceneIndexServiceBean(EntityManagerFactory EntityManagerFactory){
        LuceneIndexServiceBean luceneIndexServiceBean = new LuceneIndexServiceBean(EntityManagerFactory);
        luceneIndexServiceBean.triggerIndexing();
        return luceneIndexServiceBean;
    }

}

Finally, let us make sure that this configuration is picked up by the Spring boot application using the @Import annotation.

@EntityScan({"com.nullbeans.persistence.models"})
@SpringBootApplication
@Import(LuceneIndexConfig.class)
public class NullbeansPersistenceApplication {

..................

If you are not sure how to configure Spring beans using Java configurations, then we recommend that you check out our tutorial “How to define and declare Spring beans using Java configuration and constructor injection”.

Notice that our “Person” table is empty. Therefore, let us create a person repository and add some data to our example using a quick “CommandLineRunner” bean.

package com.nullbeans.persistence.repositories;

import com.nullbeans.persistence.models.Person;
import org.springframework.data.repository.CrudRepository;

public interface PersonRepository  extends CrudRepository<Person, Long> {
}

Let us add data using by adding the CommandLineRunner bean to our application.

@EntityScan({"com.nullbeans.persistence.models"})
@SpringBootApplication
@Import(LuceneIndexConfig.class)
public class NullbeansPersistenceApplication {

	private static final Logger log = LoggerFactory.getLogger(NullbeansPersistenceApplication.class);

	public static void main(String[] args) {
		SpringApplication.run(NullbeansPersistenceApplication.class, args);
	}


	@Bean
	public CommandLineRunner example1(PersonRepository personRepository) {

		return new CommandLineRunner() {
			@Override
			public void run(String... args) throws Exception {

				Person person = new Person();
				person.setName("My first person");
				personRepository.save(person);


			}
		};

Let us start up our Spring boot application. If everything was configured properly, then you should see the following statements in the logs

HSEARCH000027: Going to reindex 0 entities 2019-05-31 10:57:04.809  HSEARCH000028: Reindexed 0 entities

Notice that the logs says “0 entites”. This is because on startup, our only indexed entity has no data. If we re-run the application, we should see the following in the logs.

HSEARCH000027: Going to reindex 1 entities 2019-05-31 11:14:51.723  INFO 6284 HSEARCH000028: Reindexed 1 entities

You should be able to find the index in the location we defined in the application.properties file. The folder name should be the same as the fully qualified entity name. In our example, we will find the index in a folder named com.nullbeans.persistence.models.Person. The contents of the folder will look as follows.

Please keep in mind that during the uptime of your application, Hibernate Search should make sure that your index is up to data automatically. This means that you do not need to trigger the indexing everytime you create a new entity or modify an existing one. Hibernate Search will automatically re-write the affected index file.

How to query data from the Lucene index

What is the use of an index if we cannot query it, right? Let us create a query to search in our “Person” entity index. In order to search for items in a Lucene index, we will need to create a Lucene Query. Luckily, you do not need to write  complicated query syntax. We can simply use the QueryBuilder from Hibernate DSL.

The way it works is as follows:

• Instantiate a Hibernate DSL query builder. This one can generate queries which can be translated to Lucene queries.
• Use the QueryBuilder to define the query search parameters.
• Convert the query into a Lucene query using the FullTextEntityManager.
• Run the query.

Let us translate these steps into a test method.

@RunWith(SpringRunner.class)
@SpringBootTest
@Transactional
public class LuceneIndexTest {

    private static final Logger log = LoggerFactory.getLogger(LuceneIndexTest.class);

    @Autowired
    private EntityManager entityManager;

    @Test
    public void testQueryIndex() {

        //Get the FullTextEntityManager
        FullTextEntityManager fullTextEntityManager
                = Search.getFullTextEntityManager(entityManager);

        //Create a Hibernate Search DSL query builder for the required entity
        org.hibernate.search.query.dsl.QueryBuilder queryBuilder = fullTextEntityManager.getSearchFactory()
                .buildQueryBuilder()
                .forEntity(Person.class)
                .get();

        //Generate a Lucene query using the builder
        org.apache.lucene.search.Query query = queryBuilder
                .keyword()
                .onField("name").matching("first")
                .createQuery();


        org.hibernate.search.jpa.FullTextQuery fullTextQuery
                = fullTextEntityManager.createFullTextQuery(query, Person.class);


        //returns JPA managed entities
        List<Person> persons = fullTextQuery.getResultList();

        log.info("Found persons: "+ persons);
    }

}

If we run this test, we will get the following result (make sure to add a toString method to the Person class).

INFO com.nullbeans.LuceneIndexTest: Found persons: [Person{id=0, name='My first person'}]

Notice that we were able to find our entity, even though the query was not an exact match to the person’s name. This is because Lucene tokenizes the indexed data. So if your query matches on or more tokens, then you will be able to find the corresponding entity in the search results.

Summary

In this tutorial, we were able to create index JPA entity data using Lucene and Hibernate Search. We explored how to configure a Spring-boot application to utilize these search frameworks in order to index and query data.

If you have any questions, then please let us know in the comments below!