next #

Advances through the search results and returns the next page of items.

Arguments #

next();

Resolve #

Resolves to a SearchResult object, or to null if no more pages are available.

Rejects #

This method returns a rejected promise with an error if:

No pagination strategy can be applied (see below)
If invoking it would lead to more than 10 000 items being retrieved with the from/size strategy

Pagination strategies #

Depending on the arguments given to the initial search, the next method will pick one of the following strategies, by decreasing order of priority.

Strategy: scroll cursor #

If the original search query is given a scroll parameter, the next method uses a cursor to paginate results.

The results from a scroll request are frozen, and reflect the state of the index at the time the initial search request.
For that reason, this method is guaranteed to return consistent results, even if documents are updated or deleted in the database between two pages retrieval.

This is the most consistent way to paginate results, however, this comes at a higher computing cost for the server.

When using a cursor with the scroll option, Elasticsearch has to duplicate the transaction log to keep the same result during the entire scroll session.
It can lead to memory leaks if ascroll duration too great is provided, or if too many scroll sessions are open simultaneously.

Available since Kuzzle 2.2.0

You can restrict the scroll session maximum duration under the services.storage.maxScrollDuration configuration key.

try {
  const documents = [];
  for (let i = 0; i < 100; i++) {
    documents.push({ _id: `suv_no${i}`, body: { category: 'suv' } });
  }
  await kuzzle.document.mCreate('nyc-open-data', 'yellow-taxi', documents, {
    refresh: 'wait_for'
  });
  let results = await kuzzle.document.search(
    'nyc-open-data',
    'yellow-taxi',
    { query: { match: { category: 'suv' } } },
    { scroll: '10s', size: 10 });
  // Fetch the matched items by advancing through the result pages
  const matched = [];
  while (results) {
    matched.push(...results.hits);
    results = await results.next();
  }
  console.log(matched[0]);
  /*
    { _id: 'suv_no1',
      _score: 0.03390155,
      _source:
        { _kuzzle_info:
          { author: '-1',
            updater: null,
            updatedAt: null,
            createdAt: 1570093133057 },
          category: 'suv' } }
  */
  console.log(`Successfully retrieved ${matched.length} documents`);
} catch (error) {
  console.error(error.message);
}

Strategy: sort / size #

If the initial search contains sort and size parameters, the next method retrieves the next page of results following the sort order, the last item of the current page acting as a live cursor.

This strategy uses Elasticsearch search_after parameter.

You have to provide a sort combination that will always identify one item only. The recommended way is to use the field _id which is certain to contain one unique value for each document. To prevent partial retrieval of results, the SDK will reject with an error if the sort combination can identify multiple items.

Because this method does not freeze the search results between two calls, there can be missing or duplicated documents between two result pages.

This method efficiently mitigates the costs of scroll searches, but returns less consistent results: it's a middle ground, ideal for real-time search requests.

try {
  const documents = [];
  for (let i = 0; i < 100; i++) {
    documents.push({ _id: `suv_no${i}`, body: { category: 'suv' } });
  }
  await kuzzle.document.mCreate('nyc-open-data', 'yellow-taxi', documents, {
    refresh: 'wait_for'
  });
  let results = await kuzzle.document.search(
    'nyc-open-data',
    'yellow-taxi',
    {
      query: { match: { category: 'suv' } },
      sort: [
        { '_kuzzle_info.createdAt': 'desc' },
        '_id'
      ]
    },
    { size: 5 });
  // Fetch the matched items by advancing through the result pages
  const matched = [];
  while (results) {
    matched.push(...results.hits);
    results = await results.next();
  }
  console.log(matched[0]);
  /*
    { _id: 'suv_no1',
      _score: 0.03390155,
      _source:
        { _kuzzle_info:
          { author: '-1',
            updater: null,
            updatedAt: null,
            createdAt: 1570093133057 },
          category: 'suv' } }
  */
  console.log(`Successfully retrieved ${matched.length} documents`);
} catch (error) {
  console.error(error.message);
}

Strategy: from / size #

If the initial search contains from and size parameters, the next method retrieves the next page of result by incrementing the from offset.

Because this method does not freeze the search results between two calls, there can be missing or duplicated documents between two result pages.

It's the fastest pagination method available, but also the less consistent, and it is not possible to retrieve more than 10000 items using it.
Above that limit, any call to next will return a rejected promise with an error.

try {
  const documents = [];
  for (let i = 0; i < 100; i++) {
    documents.push({ _id: `suv_no${i}`, body: { category: 'suv' } });
  }
  await kuzzle.document.mCreate('nyc-open-data', 'yellow-taxi', documents, {
    refresh: 'wait_for'
  });
  let results = await kuzzle.document.search(
    'nyc-open-data',
    'yellow-taxi',
    { query: { match: { category: 'suv' } } },
    { from: 1, size: 5 });
  // Fetch the matched items by advancing through the result pages
  const matched = [];
  while (results) {
    matched.push(...results.hits);
    results = await results.next();
  }
  console.log(matched[0]);
  /*
    { _id: 'suv_no1',
      _score: 0.03390155,
      _source:
        { _kuzzle_info:
          { author: '-1',
            updater: null,
            updatedAt: null,
            createdAt: 1570093133057 },
          category: 'suv' } }
  */
  console.log(`Successfully retrieved ${matched.length} documents`);
} catch (error) {
  console.error(error.message);
}

Edit this page on Github