Auditing using Spring boot, MongoDB and JaVers

In this tutorial, we will discuss how to audit data and generating audit logs and data history entries using JaVers in a Spring boot and MongoDB environment.

Prerequisites

If you currently do not have a Spring boot environment set up with a MongoDB backend, then we suggest that you check out our Spring boot and MongoDB configuration example

This tutorial picks it up from there and extends that example so make sure you are up to date 😉

Introduction

Reasons for maintaining an audit log

In a production environment, where there are tons of data changes, an audit log, aka a data change log becomes a necessity. Reasons for maintaining an audit log include:

• Troubleshooting purposes. For example, when an error around an entity occurs, it would be great to know which change triggered a certain bug.
• Security purposes: Maintaining an audit log is extremely important when it comes to security. For example, checking who changed specific data sets, or finding out what was changed in order to investigate a security breach.
• Analytics: Tracking and analyzing data changes provides you another point of view into a user’s behavior. It allows you to learn how the user interacts and changes data in order to create new functionalities and improve existing ones.
• Data restoration: Mistakes happen. And when they do, it would be great to restore data to its original state. For example, if you make an editing mistake, then having an audit log allows you to view how data looked like before your edit and allows you to restore the data to its pre-edit state.

What is JaVers

JaVers is an open source framework that provides auditing functionality in object oriented data environments. Javers allows developers to log changes to their entities into a JaVers repository, which can later be used to perform change auditing and tracking tasks.

Why use JaVers

There are several reasons to use JaVers, including:

• It is open source and free to use.
• It is an actively maintained project with frequent updates and bug fixes. You can check the release notes here
• It is compatible with both traditional relational relational database systems as well as NoSQL systems. This makes migration between RDBMS and NoSQL much smoother.
• Easy integration with Spring and Spring Boot.

Integrating JaVers with Spring Boot and MongoDB

Integrating Javers into a Spring boot application is quite an easy task, thanks to Spring boot’s auto-configuration functionality and the existence of JaVers’ Spring boot starter module which takes advantage of the auto configuration functionality of Spring boot to get up and running very quickly.

Adding the required dependencies

In order to pull in the dependencies required to get started with JaVers in a Spring boot and MongoDB application, you will need to add the javers-spring-boot-starter-mongo dependency to your POM xml.

		<dependency>
			<groupId>org.javers</groupId>
			<artifactId>javers-spring-boot-starter-mongo</artifactId>
			<version>5.3.4</version>
		</dependency>

The dependency adds the required JaVer’s libraries and auto-configuration classes to your project. Since the auto-configuration classes are now on your classpath, Spring boot will automatically configure JaVers for you with some defaults. For the sake of simplicity of this guide, we will keep these defaults unchanged, but if you are curious, you can find more information here and here

Marking repositories for auditing

By default, JaVers will perform any auditing unless you explicitly mark your repository for auditing. To do this, you will need to use the @JaversSpringDataAuditable annotation on the repositories you would like to audit.

Remember our CarRepository from our previous tutorial ? Let us modify it with the required annotation:

@JaversSpringDataAuditable
public interface CarRepository extends MongoRepository<Car, String> {

By adding this annotation, you now create an audit entry for each create, update or delete operation you perform in the database.

Trying it out

Now, let us try out our brand new audit library configuration. For this, we will modify our CommandLineRunner application slightly by performing a modification and a deletion operation on our test data.

@Override
	public void run(String... args) throws Exception{

		//Clean the collection for our test
		carRepository.deleteAll();

		//create test data
		Car car1 = new Car("Ferrari", "2015", "488", 670);
		Car car2 = new Car("Fiat", "2012", "Abarth 595", 140);
		Car car3 = new Car("Fiat", "2007", "Abarth 500", 135);
		carRepository.save(car1);
		carRepository.save(car2);
		carRepository.save(car3);

		//find a car by model
		Car car488 = carRepository.findByModel("488");
		//modify the car
		car488.setHorsePower(800);
		carRepository.save(car488);
		log.info("Car488: {}", car488);

		//delete a car
		Car car500 = carRepository.findByModel("Abarth 500");
		carRepository.delete(car500);


		//find cars with horse power less than 200
		List<Car> cars = carRepository.findCarsWithHorsePowerLessThan(200);

		//log the results
		log.info("Found the following cars");
		for (Car foundCar: cars){
			log.info(foundCar.toString());
		}

	}

When we run our program, we will notice new log lines by our auditing framework:

org.javers.core.Javers: Commit(id:1.0, snapshots:1, author:unauthenticated, changes - NewObject:1), done in 49 millis (diff:19, persist:30)
org.javers.core.Javers: Commit(id:2.0, snapshots:1, author:unauthenticated, changes - NewObject:1), done in 7 millis (diff:2, persist:5)
org.javers.core.Javers: Commit(id:3.0, snapshots:1, author:unauthenticated, changes - NewObject:1), done in 4 millis (diff:2, persist:2)
org.javers.core.Javers: Commit(id:4.0, snapshots:1, author:unauthenticated, changes - ValueChange:1), done in 5 millis (diff:3, persist:2)
c.n.m.MongodbexampleApplication: Car: Car{id='5ca873406598431f1cb04fb7', brand='Ferrari', year='2015', model='488', horsePower=800}
org.javers.core.Javers: Commit(id:5.0, snapshots:1, author:unauthenticated, changes - ObjectRemoved:1)
c.n.m.MongodbexampleApplication: Found the following cars
c.n.m.MongodbexampleApplication: Car{id='5ca873406598431f1cb04fba', brand='Fiat', year='2012', model='Abarth 595', horsePower=140}

This is our auditing framework persisting log data into our database. If we go to our MongoDB database, we will notice that JaVers added two new collections.

The jv_head_id collection contains only a single document with the last commitId. The jv_snapshots collection contains our data change log. Here is where all the magic can be found. For each create, update or delete operation, a document in this collection can be found. There are three types of documents that can be found:

• Initial : This indicates a create operation in the database.
• Update : Indicates an update / modification operation.
• Terminate : Indicates a delete operation has been performed.

Let us check the as an example the create operation for the “488” car:

Let us highlight a few fields from our example:

• commitMetadata: This includes valuable information such as the author of the change, the time of the event in local (commitDate) and UTC (commitDateInstant). The author here is mentioned as “unauthenticated” as we did not configure any authenticion into our spring application using spring security. We will explore this in a later tutorial.
• globalId: Contains information about the entity being modified and its database id.
• state: Contains all the field values of the entity being modified. Please note that this is the state of the object after the change has been performed.
• changedProperties: This includes the fields that have been modified in this audit log entry. Since this is a creation operation, all fields have been included in the changed properties array.

Now, let us take a look at our modification operation event entry:

Notice that now, only the horsePower property has been included in the changedProperties list. Also notice that the version has been incremented to 2.

Finally, let us take a look at the deletion operation audit log entry:

Since the object has been deleted, the entry log type here is “TERMINATE”. Notice that the changed properties array is empty.

Querying the JaVers snapshots repository

Our final step here is to figure out how to access the audit logs data from our Spring boot application. For this, we simply need to obtain an instance of JaVers from the application context and search for changes for a specific car. For this, let us modify our CommandLineRunner implementation again and add a simple query to search in the JaVers snapshots collection.

For this, we will use Javers and the JaVers query builder. Make sure to import the correct classes using the following import statements:

import org.javers.core.Javers;
import org.javers.repository.jql.QueryBuilder;

In order to get the JaVers instance in our simple application, we simply need to autowire an instance of the bean.

public class MongodbexampleApplication implements CommandLineRunner
{

.... some code here ...


	@Autowired
	private Javers javers;

Now, at the end of our run method, we will add the following code. The query will try to search for all “UPDATE” snapshots that exist for the Car entity. This can be done as follows:

		QueryBuilder queryBuilder = QueryBuilder.byClass(Car.class).withSnapshotTypeUpdate();

		List<CdoSnapshot> changes = javers.findSnapshots(queryBuilder.build());

		log.info("Found the following updates: ");
		for(CdoSnapshot change: changes) {
			log.info("Car model snapshot: {}", change);
		}

If we run our program, we will get the following output:

c.n.m.MongodbexampleApplication: Found the following updates: 
c.n.m.MongodbexampleApplication: Car model snapshot: Snapshot{commit:34.0, id:...Car/5ca873406598431f1cb04fb7, version:2, (brand:Ferrari, horsePower:800, id:5ca873406598431f1cb04fb7, model:488, year:2015)}

Please note that JaVers provides a very powerful querying API. We will explore more search functionality in later tutorials.

Summary

In this tutorial, we discussed briefly what is JaVers and why it may be a good fit in a Spring boot and MongoDB stack. We then discussed how to integrate JaVers into Spring boot by adding the required dependencies and how to enable auditing for a spring repository. We then tested our example and discussed what is stored in the JaVers snapshot MongoDB collections. We then discussed a brief example for querying for JaVers entity history snapshots.