Spigot is divided into three main interfaces which are used to fetch data. The Paginator interface defines the state about our dataset such as the order information, the page size and the index of the current results. The DataProvider interface defines two methods which are used to fetch the total number of rows in the complete list of data, and the list (or a subset) of items in the dataset. The final interface is the ObjectDataset interface which extends the Paginator interface and adds methods for managing a set of results once they have been obtained from the DataProvider

public interface DataProvider<T> {
  Integer fetchResultCount();
  List<T> fetchResults(Paginator paginator);
}
                    

When we fetch the results, we pass in the paginator so the DataProvider implementation knows which rows to start returning rows from, how many rows to return and it can also set the flag on the paginator indicating whether there are more results or not.

This interface is very straightforward and its easy to see how you can implement any kind of data provider using such a simple interface. Once new data providers have been created, for the most part, they can be substituted for any other DataProvider .

We extend the DataProvider to introduce features to interface with more complex data providers. We could use the DataProvider type, but then we don't get any access to the extended data provider features without casting which ties our code to that implementation and is messy to boot. Also, by creating an interface hierarchy, we can create an implementation hierarchy to share functionality with subclasses.

Simple data providers that are in memory or file based use the basic DataProvider interface. Where the real complexity comes in is with query language based data providers. While Spigot has been designed to be expandable, it can support different query language data providers, but the main goal for this concept was accomodating JPA, Hibernate and SQL based data providers. Most of the functionality for a query language based provider is defined in the QueryDataProvider interface.

Figure 1.1. Architecture Overview

In this diagram you can see that the DataProvider subclasses define the interfaces for the multiple provider implementations. ObjectDataset and subclasses references a DataProvider implementation that is used to provide data to the dataset.

This section covers some basic examples of how each part of the core API is used. The core API consists of the Paginator , the DataProvider and the ObjectDataset interfaces. The Paginator is like a bookmark for a dataset and holds the current record index, page size and ordering information as well as flags indicating whether there is a previous or next page available. The DataProvider is the stateless part of the framework that provides the data based on the state of the paginator passed in. For each type of data store you want to access, we write a DataProvider implementation for it. The ObjectDataset is a higher level wrapper that extends the Paginator interface and adds functions to manage the results returned from a provider. The ObjectDataset has a provider attribute that it uses to fetch the data.

  DataProvider prov = new CustomDataProvider();
  Paginator paginator = new DefaultPaginator();
  List<ResultClass> results = prov.fetchResults(paginator);
  Integer count = prov.getResultCount();
                

This is a simple example where we create an instance of a custom provider that knows how to fetch the data. We create an instance of a Paginator and pass that in to the data provider when we fetch the results to tell the provider which set of data to return. In this case, we returned all the data but by setting properties on the paginator we can tell the provider just to return a subset and which row to start from. The provider is also responsible for setting the nextAvailable flag on the paginator to indicate if there are more results available.

  DataProvider<SomeClass> prov = new SomeClassDataProvider();
  Paginator paginator = new DefaultPaginator();
  paginator.setMaxRows(10);
  paginator.setFirstResult(35);
  List<SomeClass> results = prov.fetchResults(paginator);
  Integer count = prov.fetchResultCount();
                

Here we set the maximum number of rows to return to 10 and we want the first result to start at item number 35. The implementation of the SomeClassDataProvider is responsible for returning the correct subset of data to the caller when fetchResults is called.

If you were to call fetchResults a second time, the provider would go off and re-fetch the data executing whatever process it does to fetch the data. It does not cache the data and should be implemented statelessly such that one call to the provider is independent of any other calls to the provider.

The third part of the API is an ObjectDataset which combines both the provider and paginator information into one class and can be used to manage the fetched data. This component should be considered stateful as it holds on to the results and caches them. It also implements the Iterable interface to let you iterate over the dataset while taking pagination into account, so while you iterate over each item, they are loaded in a batch at a time based on the maxRows attribute. Datasets implement the Paginator interface and have the pagination attributes built in. Datasets have a provider attribute that is used to fetch the actual data so we can re-use our dataset implementation for any kind of data provider. When you change the active page, the page size or the first result returned the interal copy of the current data is invalidated and causes it to be re-fetched from the provider when it is next requested.

  DataProvider<SomeClass> provider = new SomeClassDataProvider();
  ObjectDataset<SomeClass> dataset = new DataProviderDataset(provider);
  dataset.setMaxRows(10);
  dataset.setFirstResult(35);
  List<SomeClass> results = dataset.getResults();
  
  // we can iterate over the dataset and it will automatically load the results
  // in pages of 10 results each.
  for (SomeClass item : dataset) {
    ...some code...
  }  
                

Repeated calls to getResults() without changing the pagination information will cause the exact same results to be returned since the current page of results is managed by the ObjectDataset .