Introduction

In-memory Finders

Finders can operate both in memory and with more complex data sources. Their core scenario is the latter, otherwise they could be replaced by Java 8 Stream (a more detailed comparison with Streams is at the end of this chapter); but to start with simpler code let’s have first a look at the in-memory approach.

If the whole collection of objects to query is already in memory, a predefined wrapping Finder can be created with the factory method ofCloned(); it is used by a first example implementation of PersonRegistry:

public class InMemoryPersonRegistry implements PersonRegistry
  {
    private final List<Person> persons = new ArrayList<>();

    @Override
    public void add (@Nonnull final Person person)
      {
        persons.add(person);
      }

    @Override @Nonnull
    public Finder<Person> findPerson()
      {
        return Finder.ofCloned(persons);
      }
  }

As the name of the method says, the collection is cloned (shallow clone) at construction time, so any change made after the Finder creation won’t be seen.

In-memory Finders can be useful in some real-world cases, for instance when a controller or a DAO has cached data, or to create mocks for testing classes that use more complex Finders.

Data source Finders

Now let’s see how a Finder can work with a data source that is not in memory. A classic example is the relational database, so we will use JPA (Java Persistence API) as a reference. Of course similar examples could be made with other APIs for relational database as well as with other kinds of data sources, such as NoSQL databases, semantic databases, etc.

The central class of JPA is EntityManager: it’s the facility that makes it possible to create and execute queries. What we want is make the Finder execute for us code such as:

em.createQuery(jpaql, type).setFirstResult(firstResult).setMaxResults(maxResults);

where jpaql, firstResult and maxResults have been properly set by intermediate methods previously called. In other words, if we had a test for the Finder we would like to have this kind of behaviour:

    @Test
    public void testSimpleQuery()
      {
        // when
        final List<? extends Person> results = underTest.results();
        // then
        assertThat(jpaMock.sqlQuery, is("SELECT p FROM PersonEntity p"));
        assertThat(jpaMock.firstResult, is(0));
        assertThat(jpaMock.maxResults, is(Integer.MAX_VALUE));
      }

    @Test
    public void testQueryWithAscendingSortAndFirstMax()
      {
        // when
        final List<? extends Person> results = underTest.sort(BY_FIRST_NAME)
                                                        .from(2)
                                                        .max(4)
                                                        .results();
        // then
        assertThat(jpaMock.sqlQuery, is("SELECT p FROM PersonEntity p ORDER BY p.firstName"));
        assertThat(jpaMock.firstResult, is(2));
        assertThat(jpaMock.maxResults, is(4));
      }

    @Test
    public void testQueryWithDescendingSortAndFirstMax()
      {
        // when
        final List<? extends Person> results = underTest.sort(BY_LAST_NAME, DESCENDING)
                                                        .from(3)
                                                        .max(7)
                                                        .results();
        // then
        assertThat(jpaMock.sqlQuery, is("SELECT p FROM PersonEntity p ORDER BY p.lastName DESC"));
        assertThat(jpaMock.firstResult, is(3));
        assertThat(jpaMock.maxResults, is(7));
      }

    @Test
    public void testQueryWithDoubleSort()
      {
        // when
        final List<? extends Person> results = underTest.sort(BY_LAST_NAME, DESCENDING)
                                                        .sort(BY_FIRST_NAME, ASCENDING)
                                                        .results();
        // then
        assertThat(jpaMock.sqlQuery, is("SELECT p FROM PersonEntity p ORDER BY p.lastName DESC, p.firstName"));
        assertThat(jpaMock.firstResult, is(0));
        assertThat(jpaMock.maxResults, is(Integer.MAX_VALUE));
      }

    @Test
    public void testQueryWithCount()
      {
        // when
        final int count = underTest.count();
        // then
        assertThat(jpaMock.sqlQuery, is("SELECT COUNT(p) FROM PersonEntity p"));
        assertThat(jpaMock.firstResult, is(0));
        assertThat(jpaMock.maxResults, is(Integer.MAX_VALUE));
      }

Before going on, let’s consider that transactions are managed by JPA in a few ways that, while not particularly complex in the context of a real application, require excessive set up for a simple example like the one we’re dealing with. So we introduce a simple helper that executes a task in the context of a transaction:

    public <T> T computeInTx (@Nonnull Function<EntityManager, T> task);

    public default void runInTx (@Nonnull final Consumer<EntityManager> task)

In a real case the EntityManager would rather be injected.

The first thing we need is to define the state of the Finder, which must both model the parameters set by intermediate methods and contain a reference to the data source (which, in our case, is TxManager).

    @Nonnull
    private final TxManager txManager;

    @Nonnegative
    private final int firstResult;

    @Nonnegative
    private final int maxResults;

    @Nonnull
    private final List<Pair<JpaqlSortCriterion, SortDirection>> sortCriteria;

Let’s now focus on the implementation of intermediate methods. They usually don’t do anything smart, but just accumulate the required parameters for later performing the query. Since a Finder must be immutable, they can’t change the internal state: they rather must create and return a cloned Finder with the original state and only a single field changed. This is a typical approach for immutable objects.

    @Override @Nonnull
    public Finder<Person> from (@Nonnegative final int firstResult)
      {
        return new JpaPersonFinder(txManager, firstResult, maxResults, sortCriteria);
      }

    @Override @Nonnull
    public Finder<Person> max (@Nonnegative final int maxResults)
      {
        return new JpaPersonFinder(txManager, firstResult, maxResults, sortCriteria);
      }

Now let’s deal with sorting. Sorting works in a different way in function of the Finder being “in memory” or associated to a data source:

• in-memory happens by means of the sorting features of the Java Collection API, so there’s nothing special about it; but it is to be pointed out that it is performed before pagination, so it’s quite different calling from()/max() » results() » sorting and sort()/from()/max() » results()
• in the data source, which allows to optimize the query (if the data source cooperates).

In both cases sorting criteria are defined by means of the interfaces SortCriterion and InMemorySortCriterion, which extends the former. InMemorySortCriterion declares a method which will be called by the Finder to perform the sort:

public void sort (@Nonnull List<? extends TYPE> results, @Nonnull SortDirection sortDirection);

A convenience method of() makes it possible to easily create a working SortCriterion by wrapping a Comparator:

    public static final SortCriterion BY_FIRST_NAME = InMemorySortCriterion.of(comparing(Person::getFirstName));

    public static final SortCriterion BY_LAST_NAME = InMemorySortCriterion.of(comparing(Person::getLastName));

The intermediate method Finder.sort() behaves as other intermediate methods and just collects data for a later use:

    @Override @Nonnull
    public Finder<Person> sort (@Nonnull final SortCriterion criterion, @Nonnull final SortDirection direction)
      {
        if (!(criterion instanceof JpaqlSortCriterion))
          {
            throw new IllegalArgumentException("Can't sort by " + criterion);
          }

        return new JpaPersonFinder(txManager,
                                   firstResult,
                                   maxResults,
                                   concat(sortCriteria, Pair.of((JpaqlSortCriterion)criterion, direction)));
      }

Note that it usually rejects implementations of SortCriterion that it doesn’t know.

While the implementation of SortCriterion could be a simple enum that is later evaluated in a switch, in a good design it provides its own behaviour (which is disclosed only to the Finder implementation). In case of JPA is to assemble the ORDER BY section of the query:

    @RequiredArgsConstructor
    static final class JpaqlSortCriterion implements SortCriterion
      {
        @Nonnull
        private final String field;

        @Nonnull
        public String processSql (@Nonnull final String jpaql, @Nonnull final SortDirection sortDirection)
          {
            final String orderBy = jpaql.contains("ORDER BY") ? ", " : " ORDER BY ";
            return jpaql + orderBy + field + ((sortDirection == SortDirection.DESCENDING) ? " DESC" : "");
          }
      }
    
    public static final SortCriterion BY_FIRST_NAME = new JpaqlSortCriterion("p.firstName");
    public static final SortCriterion BY_LAST_NAME  = new JpaqlSortCriterion("p.lastName");

The core part of the Finder is where it finalises and executes the query. It creates the JPAQL query and then it callsEntityManager to execute it.

@Nonnull
private <S> TypedQuery<S> createQuery (@Nonnull final EntityManager em,
                                       @Nonnull final Class<S> type,
                                       @Nonnull final String jpaqlPrefix)
  {
    final AtomicReference<String> temp = new AtomicReference<>(jpaqlPrefix + " FROM PersonEntity p");
    sortCriteria.forEach(p -> temp.set(p.a.processSql(temp.get(), p.b)));
    final String jpaql = temp.get();
    log.info(">>>> {}", jpaql);
    return
            // START SNIPPET: createQuery
            em.createQuery(jpaql, type).setFirstResult(firstResult).setMaxResults(maxResults);
            // END SNIPPET: createQuery
  }

At last we can implement termination methods: they run the query, extract the part of the results they need and convert them from a JPA entity to the desired class (this task may be needed or not in function of the architecture of the application: a Finder might expose JPA entities if desired).

    @Override @Nonnull
    public Optional<Person> optionalResult()
      {
        final List<? extends Person> results = results();

        if (results.size() > 1)
          {
            throw new RuntimeException("More than a single result");
          }

        return (Optional<Person>)results.stream().findFirst();
      }

    @Override @Nonnull
    public Optional<Person> optionalFirstResult()
      {
        // Warning: the stream must be consumed *within* runInTx2()
        return txManager.computeInTx(em -> createQuery(em, PersonEntity.class, "SELECT p")
                .getResultStream()
                .findFirst()
                .map(JpaPersonRegistry::fromEntity));
      }

    @Override @Nonnull
    public List<? extends Person> results()
      {
        // Warning: the stream must be consumed *within* runInTx2()
        return txManager.computeInTx(em -> createQuery(em, PersonEntity.class, "SELECT p")
                .getResultStream()
                .map(JpaPersonRegistry::fromEntity)
                .collect(Collectors.toList()));
      }

    @Override @Nonnegative
    public int count()
      {
        return txManager.computeInTx(em -> createQuery(em, Long.class, "SELECT COUNT(p)").getSingleResult()).intValue();
      }

A point that is worth mentioning is about how transactions are handled: it largely depends on the used technology, as one needs to respect the best or mandatory practices that come with it. In the case of JPA, it is required that the Stream of results produced by a query is consumed before the transaction is committed; in our case this means within the call to TxManager.

Extended Finders

An extended Finder is a subclass of Finder that exposes additional methods for filtering the results. For instance we could write a PersonFinder for the previous PersonRegistry that extends Finder<Person> and offers two new methods that filter by first or last name with a regular expression:

    @Nonnull
    public PersonFinder withFirstName (@Nonnull String regex);

    @Nonnull
    public PersonFinder withLastName (@Nonnull String regex);

The registry now would return a PersonFinder instead of the general Finder<Person>, like this:

public interface PersonRegistry2 extends PersonRegistry
  {
    @Override @Nonnull
    public PersonFinder findPerson();
  }

There is a first problem to address: to make it possible to freely mix all the intermediate methods, both the new ones and those defined in the base Finder. This cannot be achieved by merely extending the Finder interface (i. e. interface PersonFinder extends Finder<Person>), as the methods declared in Finder return a value which is statically typed as Finder; so the compiler would not allow to call the new methods. In other words this would be possible:

    List<Person> persons = findPerson().withLastName("B.*").max(5).results();

but this wouldn’t compile:

    List<Person> persons = findPerson().max(5).withLastName("B.*").results();

Free mixing of methods is mandatory to fulfill the flexibility target that allows a portion of the application to refine a query that has been partially constructed in another part of the application.

To address this problem a specific interface named ExtendedFinderSupport is provided. It just re-declares the methods provided by Finder by overriding their return value type (in our example to PersonFinder in place of Finder<Person>). This is possible thanks to the fact that Java features covariant return type.

ExtendedFinderSupport takes two generics: the type of the managed object (Person) and type of the new Finder (PersonFinder). To better understand this, have a look at theExtendedFinderSupport source:

public interface ExtendedFinderSupport<TYPE, EXTENDED_FINDER extends Finder<TYPE>> extends Finder<TYPE>
  {
    /** {@inheritDoc} */
    @Override @Nonnull
    public EXTENDED_FINDER from (@Nonnegative int firstResult);

    /** {@inheritDoc} */
    @Override @Nonnull
    public EXTENDED_FINDER max (@Nonnegative int maxResults);

    /** {@inheritDoc} */
    @Override @Nonnull
    public EXTENDED_FINDER sort (@Nonnull SortCriterion criterion);

    /** {@inheritDoc} */
    @Override @Nonnull
    public EXTENDED_FINDER sort (@Nonnull SortCriterion criterion, @Nonnull SortDirection direction);

    /** {@inheritDoc} */
    @Override @Nonnull
    public EXTENDED_FINDER withContext (@Nonnull Object context);
  }

So a properly designed PersonFinder must extend ExtendedFinderSupport<Person, PersonFinder> in place of Finder<Person>:

public interface PersonFinder extends ExtendedFinderSupport<Person, PersonFinder>
  {
    // START SNIPPET: new-methods
    @Nonnull
    public PersonFinder withFirstName (@Nonnull String regex);

    @Nonnull
    public PersonFinder withLastName (@Nonnull String regex);
    // END SNIPPET: new-methods
  }

In this way the new methods can be freely mixed with the ones inherited by the super interface:

Error during retrieving content skip as ignoreDownloadError activated.

Hierarchic Finders

In a complex application it might be convenient to write a number of different Finders in form of a hierarchy, for instance because there is some common behaviour that can be effectively captured by means of the generalisation-specialisation relationship (even though composition often is a better approach). The Finder API doesn’t mandate anything in addition of respecting the contract declared in its interface and have an immutable implementation, so one can proceed with his favourite design strategy. Anyway the API provides a support class FinderSupport which offers the capability of having a completely encapsulated status: that is with all fields private (rather than protected) and each level of the hierarchy doesn’t know anything of the internal status of the others. This is a way to mitigate the tight coupling caused by inheritance, so one can make changes to the internal status to a Finder in an intermediate level of the hierarchy without forcing the subclasses to be adjusted.

To explain how this works by examples, we are going to show how an implementation of the extended Finder we introduced in the previous section might be done (in-memory, to keep things simpler).

First we have to declare fields for the internal state and a public constructor to initialize the object with reasonable defaults:

    @Nonnull
    private final List<Person> persons;

    @Nonnull
    private final String firstNameRegex;

    @Nonnull
    private final String lastNameRegex;

    // This is for public use
    public PersonFinderImpl2 (@Nonnull final List<Person> persons)
      {
        this(persons, ".*",  ".*");
      }

A private constructor to initialize everything to arbitrary values is also needed:

// This could be generated by Lombok's @RequiredArgsConstructor
private PersonFinderImpl2 (@Nonnull final List<Person> persons,
                           @Nonnull final String firstNameRegex,
                           @Nonnull final String lastNameRegex)
  {
    this.persons = persons;
    this.firstNameRegex = firstNameRegex;
    this.lastNameRegex = lastNameRegex;
  }

As it was explained above, intermediate methods must create copies of the Finder to comply with the immutability constraint. In a normal class this would be performed by a copy constructor that takes all the fields, including those of the superclass(es); but since we decided to make them private they can’t be accessed. So all we can do is to call the constructor shown in the above code snippet that only deals with the fields of the current class. Since it calls the super default constructor, this means that the state of the super class(es) will be reset to a default: i.e. any change applied by intermediate methods implemented in the super class(es) will be lost. Obviously this is not how things are supposed to work: that’s why FinderSupport offers a clonedWithOverride() method that fixes everything.

    @Override @Nonnull
    public PersonFinder withFirstName (@Nonnull final String regex)
      {
        return clonedWith(new PersonFinderImpl2(persons, regex, lastNameRegex));
      }

    @Override @Nonnull
    public PersonFinder withLastName (@Nonnull final String regex)
      {
        return clonedWith(new PersonFinderImpl2(persons, firstNameRegex, regex));
      }

How does it work? It relies on the the presence of a special copy constructor that looks like this:

public PersonFinderImpl2 (@Nonnull final PersonFinderImpl2 other, @Nonnull final Object override)
  {
    super(other, override);
    final PersonFinderImpl2 source = getSource(PersonFinderImpl2.class, other, override);
    this.persons = source.persons;
    this.firstNameRegex = source.firstNameRegex;
    this.lastNameRegex = source.lastNameRegex;
  }

Note: having this special copy constructor is a requirement of any subclass of FinderSupport. The FinderSupport constructor makes a runtime check by introspection and throws an exception if the proper copy constructor is not found.

It takes two parameters:

1. other is the usual parameter used in a clone constructor and references the instance being cloned.
2. override is the incomplete finder we instantiated in our custom intermediate methods. It holds the variations to apply to the state of the new Finder.

We need to initialize all the fields of our pertinence (that is, the ones declared in the current class) choosing from where to get their values. Aren’t they in the override object? No, they aren’t always there. If we are in a hierarchy of Finders all copy constructors will be called wherever a change is made; in other words, we aren’t sure that our portion of state is the one that needs to be partially changed. We can tell by looking at the dynamic type of the override object: if it is our same type, it’s the incomplete Finder with the new values, and we must initialize from it. Otherwise we must initialize as in a regular clone constructor, from the other object. A convenience method getSource() performs the decision for us. Of course we need to call the super() constructor to make sure everything is fine (but no details of the super class are exposed by it).

Is it a bit clumsy? Admittedly it is, even though the code is simple and clean: once the concept is clear, it’s easy to write a copy constructor for a new extended Finder. Part of the clumsiness derives from the complexity of inheritance, that we are trying to work around. If you don’t like this approach, just forget FinderSupport.

Note: if you are perplexed by the concept of “incomplete” Finder (which is a curious thing indeed, a short-lived object “degraded“ to a value object) you can use a simpler value object just holding the required values. Since override is a generic Object, it will work. Again, this approach requires some more code to write; but here @Data annotation from Lombok or Java 16 records might be useful.*

Marginal note to FinderSupport

Note: this part of the API might go away in future: mention that after TFT-262 a Finder implementation only requires results().

If you decide to implement a Finder by subclassing FinderSupport there is an alternative way to implement the termination methods, as they have default implementations. You can rather implement either of these two methods:

@Nonnull
protected List<? extends TYPE> computeNeededResults()

This method is responsible to produce the final results as they will be returned to the caller. That is it must respect parameters concerning pagination (from() or max()), sorting and such. For instance, if the source is a relational database this method should prepare and execute a SQL query with all the relevant clauses (WHERE, ORDER BY, LIMIT, etc.). If this method is not overridden, it will call the method shown below and then apply pagination and sorting by itself (in memory).

@Nonnull
protected List<? extends TYPE> computeResults()

This method would return all the objects of pertinence, without filtering or sorting them; the default implementation of computeNeededResults() will take care of that. Since this implies to work in memory after having loaded/created all the objects, this approach is easier to write but less efficient. It’s ok for mocks or simple cases. The implementation of our example is:

    @Override @Nonnull
    protected List<? extends Person> computeResults()
      {
        final Pattern firstNamePattern = Pattern.compile(firstNameRegex);
        final Pattern lastNamePattern = Pattern.compile(lastNameRegex);

        return persons.stream()
                      .filter(p -> firstNamePattern.matcher(p.getFirstName()).matches()
                                && lastNamePattern.matcher(p.getLastName()).matches())
                      .collect(Collectors.toList());
      }

Comparison with Java 8 Stream

A first look at Finder, in particular the presence of intermediate and termination methods, sure recalls a similarity with Java 8 Stream. Finder was designed before Java 8 existed and at that time it partly covered functions that were later made available with Stream; but it has been conceived with a different scope:

• Stream is a library facility that focuses on a functional and efficient way to navigate through an abstract sequence of objects; it can be customised via Spliterator for integrating to unusual data sources, but it can’t interact with them. In other words, a Spliterator can’t receive from the Stream information about filtering or sorting: first data are extracted from the data source, then they are manipulated in memory. Last but not least, the API has a predefined set of exposed methods that can’t be extended.
• Finder, instead, is a business facility that can interact with the data source and is well aware of the business model; so it can be extended with new methods that are related to the specific structure of model classes (in the previous example, by knowing that a Person has firstName and lastName).

Furthermore it has been designed to integrate with another member of this library, which is named As and allows to use a particular implementation of the DCI architectural pattern.

A Stream can filter results by means of function composition: for instance filter(p -> Pattern.matches("B.*", p.getFirstName())); but in this case filtering happens only after the objects have been loaded in memory because the data source has no way to know what is happening and cannot optimise its behaviour. For instance, if the data source is a DAO to a database, it can’t create an ad-hoc SQL statement; Finder instead can cooperate with the data source and prepare an optimised query.

Finders can be effectively be used in synergy with Stream by chaining the appropriated methods: this allows to choose which part of the processing must be performed by the data source and which part in memory, after data have been retrieved.

Error during retrieving content skip as ignoreDownloadError activated.

This explains why Finder doesn’t offer methods such as filter(Predicate<T>): because in no way from a compiled Java function it could understand how to prepare a query for a generic data source. Such a method would be only useful to post-process data once they have been loaded in memory, but it’s more effective to pass the results to a Stream and use the standard Java API.