v4.1.0

V4.1.0 Release Notes

This release marks a version bump within the 4.x branch committed to Laravel 10 & 11 compatibility.

Acknowledgements

This release represents a significant advancement in the package's development, primarily driven by PR #32. Key improvements include:

• Comprehensive refactoring of the package
• Introduction of new features
• Extensive coverage with automated tests
• Enhanced compliance with coding standards and type-checking

A sincere thank you to @use-the-fork for their substantial contributions to this release. Their efforts have been instrumental in elevating the quality, reliability and maturity of this package.

Breaking Changes

There's only one breaking change:

$query->orderBy(string $column, string $direction = 'asc', string $mode = null, array $missing = '_last');
// Has been changed back to:
$query->orderBy(string $column, string $direction = 'asc');
// With a new method to add any ES sort parameter to a given column:
$query->withSort(string $col, string  $key, mixed $value);

• This also applies to orderByDesc

Examples

$products = Product::orderBy('color.keyword', 'desc', null, '_first')->get();
// Is now:
$products = Product::orderBy('color.keyword', 'desc')->withSort('color.keyword', 'missing', '_first')->get();

And:

$products = Product::where('is_active', true)->orderBy('order_values', 'desc', 'avg')->get();
// Is now:
$products = Product::where('is_active', true)->orderBy('order_values', 'desc')->withSort('order_values', 'mode', 'avg')->get();

This change is to future-proof for the outlier sorting options that ES offers, ex:

$products = Product::withSort('price', 'unmapped_type', 'long')->get();
$products = Product::withSort('price', 'ignore_unmapped', true')->get();

Inspired by #36 via @ildar-developer

Refactor

• Automated tests
• PHPStan compliance
• PINT formatted
• Updated PHPDocs for better IDE support

New Features

1. Cursor Pagination

Given the default limitations of Elasticsearch to paginate beyond 10k records, cursor pagination allows you to paginate indefinitely using search_after given a sort map.

As is the case with core Laravel's cursorPaginate , you can only page next and prev

cursorPaginate($perPage, $columns, $cursorName, $cursor)

Example:

Product::orderByDesc('orders')->orderByDesc('price')->cursorPaginate(50)->withQueryString();

The accuracy of this method depends on the sort, as would be the case using ES directly. If no sort is provided, it will try to sort by created_at and updated_at if those mappings are found; otherwise, it will throw a MissingOrderException.

---

2. Bulk Import

insert($values, $returnData = null);
insertWithoutRefresh($values, $returnData = null);

These methods use Elasticsearch's Bulk API under the hood, providing efficient ways to insert multiple documents at once.

If the $returnData parameter is null or false, it will return a summary as an array:

{
    "success": true,
    "timed_out": false,
    "took": 7725,
    "total": 100000,
    "created": 100000,
    "modified": 0,
    "failed": 0
}

Otherwise, it will return an ElasticCollection of all the inserted records.

Performance and Usage:

1. insert($values, $returnData = null)
   • Performs a bulk insert and waits for the index to refresh.
   • Ensures that inserted documents are immediately available for search.
   • Use this when you need the inserted data to be searchable right away.
   • Slower than insertWithoutRefresh but provides immediate consistency.
2. insertWithoutRefresh($values, $returnData = null)
   • Executes bulk inserts without waiting for the index to refresh.
   • Offers a significant speed boost compared to insert().
   • The speed increase is due to skipping the index refresh operation, which can be resource-intensive.
   • Inserted records may not be immediately available for search
   • Use this when you need to insert large amounts of data quickly and can tolerate a slight delay in searchability.

When to use each:

1. Use insert() when:
   • You need immediate searchability of inserted data.
   • You're inserting smaller batches of data where the performance difference is negligible.
   • In user-facing applications where real-time data availability is crucial.
2. Use insertWithoutRefresh() when:
   • You're performing large batch imports where speed is a priority.
   • In background jobs or data migration scripts where immediate searchability isn't necessary.
   • You can handle a delay between insertion and searchability in your application logic.

---

3. ElasticCollection

Queries that return collections (get(), search(), insert()) now return them as an ElasticCollection. ElasticCollection is the same as Laravel's Eloquent Collection but with the executed query's metadata embedded in it.

$fetch = Product::where('orders', '>', 100)->limit(50)->orderByDesc('price')->get();
$shards = $fetch->getShards(); // Shards info
$dsl = $fetch->getDsl(); // The full query DSL that was used
$total = $fetch->getTotal(); // Total records in the index if there were hits
$maxScore = $fetch->getMaxScore(); // Max score of the search
$took = $fetch->getTook(); // Time taken for the search in milliseconds
$meta = $fetch->getQueryMetaAsArray(); // All the metadata in an array

return $fetch; //gives your collection of results as always

Bug fix

• Starting a new query before an existing one has been executed was causing the connection (and queries) to get mixed up. via #36

What's Changed

• Refract/cleanup - Added Test suite / formating etc. by @use-the-fork in #32

Full Changelog: v4.0.4...v4.1.0