As a Laravel developer, you’ve probably faced the challenge of implementing robust search functionality in your application. You may have struggled with slow query performance, limited filtering options, and difficulty customizing search results to meet specific use cases.
You’ll build an AI-powered search feature that can handle complex queries, provide accurate and relevant results, and adapt to changing user behavior. By following this tutorial, you’ll learn how to integrate Elasticsearch and OpenAI into your Laravel application, implementing a search function that’s not only fast but also contextually aware and customizable.
Setting Up the Project Structure for AI Search
To implement AI-powered search in Laravel, we need a solid project structure that separates concerns and allows for easy maintenance and extension. Let’s create a new Laravel project using Composer:
composer create-project --prefer-dist laravel/laravel ai-search-appNavigate into the newly created directory:
cd ai-search-appCreate a new database migration for the table that will hold the source content we’ll later index into Elasticsearch:
// database/migrations/2023_03_01_000000_create_search_index_table.php
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
return new class extends Migration
{
public function up(): void
{
Schema::create('search_index', function (Blueprint $table) {
$table->id();
$table->string('title');
$table->text('description');
// Add other fields as necessary for your search index
});
}
public function down(): void
{
Schema::dropIfExists('search_index');
}
};Run the migration to create the search_index table:
php artisan migrateCreate a new service provider with php artisan make:provider SearchServiceProvider. It will register a shared Elasticsearch client that the rest of our search code can type-hint:
// app/Providers/SearchServiceProvider.php
namespace App\Providers;
use Elastic\Elasticsearch\Client;
use Elastic\Elasticsearch\ClientBuilder;
use Illuminate\Support\ServiceProvider;
class SearchServiceProvider extends ServiceProvider
{
public function register(): void
{
$this->app->singleton(Client::class, function () {
return ClientBuilder::create()
->setHosts([config('services.elasticsearch.host')])
->build();
});
}
}This sets the foundation for our AI search implementation. In the next section, we’ll install required libraries and configure Laravel’s Elasticsearch integration.
Installing Required Libraries: Elasticsearch and OpenAI
To implement AI-powered search in Laravel, we need to install two essential libraries: the official Elasticsearch PHP client and the OpenAI PHP client for Laravel.
First, let’s install the Elasticsearch client using Composer:
composer require elasticsearch/elasticsearchThis library lets our application create indices, store documents, and run search queries against Elasticsearch.
Next, we’ll install OpenAI using Composer. We’ll use the openai-php/laravel package to interact with the OpenAI API:
composer require openai-php/laravelMake sure you have an OpenAI account and an API key, as we’ll need it later in this tutorial. If you haven’t already, create a new file named .env in your project’s root directory and add the following line to store your API key:
OPENAI_API_KEY=YOUR_OPENAI_API_KEY_HEREReplace YOUR_OPENAI_API_KEY_HERE with your actual OpenAI API key.
Now that we have installed both libraries, let’s configure them in our Laravel project. We’ll start by running the OpenAI package’s install command, which publishes the config/openai.php configuration file and adds the API key variables to your .env file:
php artisan openai:installWe’ll then update the .env file with the OpenAI API key; the package reads it from OPENAI_API_KEY automatically.
With these libraries installed, we’re now ready to move on to configuring Elasticsearch integration in our Laravel application.
Configuring Laravel’s Elasticsearch Integration
To take advantage of Elasticsearch’s powerful search capabilities within our Laravel application, we need to configure its integration with our project.
Firstly, ensure that you have installed the elasticsearch/elasticsearch package using Composer. The client’s major version should match your Elasticsearch server’s major version, so pin it explicitly; for an 8.x cluster that looks like this (use "^9.0" for a 9.x cluster):
composer require elasticsearch/elasticsearch "^8.12"The official client doesn’t ship a Laravel service provider, so we’ll use the SearchServiceProvider we created earlier. php artisan make:provider registers it automatically in bootstrap/providers.php on Laravel 11 and later; make sure it’s listed there (or in the providers array of config/app.php on older versions):
// bootstrap/providers.php
return [
App\Providers\AppServiceProvider::class,
App\Providers\SearchServiceProvider::class,
];We also need to add the connection settings to the config/services.php file:
// config/services.php
'elasticsearch' => [
'host' => env('ELASTICSEARCH_HOST', 'http://localhost:9200'),
],Here, we can configure various Elasticsearch settings such as our cluster URL and credentials.
For example, let’s assume our Elasticsearch instance is hosted on https://my-elasticsearch-cluster:9200. We would set the ELASTICSEARCH_HOST value in our .env file accordingly:
ELASTICSEARCH_HOST=https://my-elasticsearch-cluster:9200Lastly, we’ll add a simple route to test our Elasticsearch integration. Open up routes/web.php and append the following code:
use Elastic\Elasticsearch\Client;
Route::get('/search', function (Client $client) {
// Returns the cluster name and version if the connection works
return $client->info()->asArray();
});This route resolves the client from the container and asks our Elasticsearch instance for its basic cluster information (name and version). If the connection or credentials are wrong, the client throws an exception instead. We can now navigate to this route in our web browser (e.g., http://localhost:8000/search) to verify that our integration is working as expected.
With this section complete, we have successfully configured Laravel’s Elasticsearch integration within our project.
Implementing the AI-Powered Search Functionality using OpenAI
Implementing AI-Powered Search Functionality using OpenAI
With Elasticsearch set up and connected to our Laravel application, it’s time to integrate OpenAI into our search functionality. We’ll use the openai-php/laravel library to interact with the OpenAI API.
First, we need to install the openai-php/laravel library via Composer (skip this if you installed it earlier):
composer require openai-php/laravelOpenAI doesn’t offer a search endpoint; instead, we’ll use its embeddings API to turn text into vectors and let Elasticsearch find the closest matches. Next, we’ll create a new service class to handle interactions with the OpenAI API. Create a new file at app/Services/OpenAiService.php:
// app/Services/OpenAiService.php
namespace App\Services;
use OpenAI\Laravel\Facades\OpenAI;
class OpenAiService
{
public function embed(string $text): array
{
$response = OpenAI::embeddings()->create([
'model' => 'text-embedding-3-small',
'input' => $text,
]);
return $response->embeddings[0]->embedding;
}
}In this example, we’re using the OpenAI facade to call the embeddings endpoint with the text-embedding-3-small model. It returns an array of 1,536 numbers that captures the meaning of the text, so texts with similar meanings produce similar vectors.
Before we can search, every document needs an embedding stored in Elasticsearch. Add an Artisan command to routes/console.php that creates an articles index with a dense_vector field and indexes the rows from our search_index table:
// routes/console.php
use App\Services\OpenAiService;
use Elastic\Elasticsearch\Client;
use Illuminate\Support\Facades\Artisan;
use Illuminate\Support\Facades\DB;
Artisan::command('search:index', function (Client $client, OpenAiService $openAi) {
// Create the index with a dense_vector field for the embeddings
$client->indices()->create([
'index' => 'articles',
'body' => [
'mappings' => [
'properties' => [
'title' => ['type' => 'text'],
'description' => ['type' => 'text'],
'embedding' => [
'type' => 'dense_vector',
'dims' => 1536,
'index' => true,
'similarity' => 'cosine',
],
],
],
],
]);
// Index every row from the search_index table along with its embedding
foreach (DB::table('search_index')->get() as $row) {
$client->index([
'index' => 'articles',
'id' => $row->id,
'body' => [
'title' => $row->title,
'description' => $row->description,
'embedding' => $openAi->embed($row->title . ' ' . $row->description),
],
]);
}
$this->info('Indexing complete.');
});Run php artisan search:index to build the index.
Now that we have our service class set up, let’s integrate it into our search functionality in Laravel. We’ll modify the search controller method to use the OpenAiService and run a kNN (nearest neighbour) search in Elasticsearch:
// app/Http/Controllers/SearchController.php
namespace App\Http\Controllers;
use App\Services\OpenAiService;
use Elastic\Elasticsearch\Client;
use Illuminate\Http\Request;
class SearchController extends Controller
{
public function search(Request $request, OpenAiService $openAi, Client $client)
{
// Turn the user's query into an embedding vector
$vector = $openAi->embed($request->input('query'));
// Find the documents whose embeddings are closest to the query
$response = $client->search([
'index' => 'articles',
'body' => [
'knn' => [
'field' => 'embedding',
'query_vector' => $vector,
'k' => 10,
'num_candidates' => 100,
],
'_source' => ['title', 'description'],
],
]);
$results = $response['hits']['hits'];
// ...
}
}This is a basic implementation of AI-powered search using OpenAI embeddings and Elasticsearch. In the next section, we’ll explore how to fine-tune our search results with custom scoring and ranking.
Tuning Search Results with Custom Scoring and Ranking
Now that we have our AI-powered search functionality up and running, let’s focus on fine-tuning the results to better suit our application’s needs. By default, Elasticsearch scores kNN results purely by how similar each document’s vector is to the query vector. However, there may be cases where you want to override or modify these scores based on specific requirements.
One common use case is custom scoring for certain types of content. For example, in an e-commerce application, you might want to boost the visibility of products with high customer ratings or new releases.
To achieve this, we can combine the kNN search with a regular query wrapped in Elasticsearch’s function_score query. Elasticsearch adds the scores of both parts (each weighted by its boost), and the field_value_factor function factors a numeric field such as customer_rating into the score of each matching document (this assumes your indexed documents include that field).
// Inside SearchController::search(), after computing $vector
$query = $request->input('query');
$response = $client->search([
'index' => 'articles',
'body' => [
// Semantic matches from the embedding vector
'knn' => [
'field' => 'embedding',
'query_vector' => $vector,
'k' => 10,
'num_candidates' => 100,
'boost' => 0.5,
],
// Keyword matches, boosted by customer rating
'query' => [
'function_score' => [
'query' => ['match' => ['description' => $query]],
'field_value_factor' => [
'field' => 'customer_rating',
'factor' => 0.2,
'missing' => 0,
],
'boost_mode' => 'sum',
],
],
],
]);Another aspect of search result tuning is ranking. By default, Elasticsearch uses a best-match-first approach, returning results in descending order of score. However, you can specify a custom ranking strategy by passing a sort clause.
$query = $request->input('query');
$response = $client->search([
'index' => 'articles',
'body' => [
'query' => ['match' => ['description' => $query]],
// Rank newer releases first, then by relevance
'sort' => [
['release_date' => ['order' => 'desc', 'missing' => '_last']],
'_score',
],
],
]);By leveraging custom scoring and ranking strategies, you can further refine your search results to better meet the needs of your application. This will be crucial in our next section when we integrate these search results with our application’s UI.
Integrating Search Results with Your Application’s UI
Now that our AI-powered search functionality is implemented and tuned, it’s time to integrate the results into our application’s user interface. For this example, let’s assume we’re working on a blog platform where users can search for articles.
First, create a new Blade template for displaying search results: resources/views/search/results.blade.php.
<!-- resources/views/search/results.blade.php -->
<div class="search-results">
@foreach($results as $result)
<div>
{{ $result['_source']['title'] }} ({{ $result['_score'] }})
<a href="{{ route('article.show', ['id' => $result['_id']]) }}">Read More</a>
</div>
@endforeach
</div>In this template, we’re looping through the search results and displaying each article’s title, score (which represents the relevance of the result), and a link to read more. The link assumes your application already defines an article.show route that accepts the article’s ID; adjust it to match your own routes.
Next, update your search controller to render the layout, which includes the results.blade.php template, and to show only the search form when no query has been entered: app/Http/Controllers/SearchController.php.
// app/Http/Controllers/SearchController.php
public function search(Request $request, OpenAiService $openAi, Client $client)
{
// Show just the search form until the user has entered a query
if (! $request->filled('query')) {
return view('layouts.app');
}
// ... (previous implementation remains the same)
return view('layouts.app', ['results' => $results]);
}Then register the route for the controller in routes/web.php, replacing the test route from earlier:
use App\Http\Controllers\SearchController;
Route::get('/search', [SearchController::class, 'search'])->name('search.index');Finally, update your application’s layout to include a search form and display the results: resources/views/layouts/app.blade.php.
<!-- resources/views/layouts/app.blade.php -->
<div class="container">
<div class="row justify-content-center">
<div class="col-md-8">
<h1>Search</h1>
<form method="GET" action="{{ route('search.index') }}">
<input type="text" name="query" placeholder="Search...">
<button type="submit">Search</button>
</form>
@isset($results)
@include('search.results')
@endisset
</div>
</div>
</div>With these changes, users can now search for articles and see the relevant results displayed on the page. This concludes our tutorial on implementing AI-powered search in Laravel!
Fine-Tuning and Deploying the AI Search Feature
Now that you’ve implemented AI-powered search in your Laravel application, it’s essential to fine-tune its performance. This involves tweaking settings, adjusting parameters, and monitoring metrics to ensure seamless user experience.
Start by analyzing search logs to identify areas for improvement. You can use tools like Laravel Telescope or third-party services like Datadog. This will help you pinpoint bottlenecks in your search pipeline, such as slow query execution times or high memory usage.
To optimize performance, you may need to adjust OpenAI’s model settings, Elasticsearch configurations, or even tweak your database schema. For example:
// Request shorter embeddings to save storage and speed up kNN search.
// The 'dims' value in the index mapping must match, so reindex after changing it.
$response = OpenAI::embeddings()->create([
'model' => 'text-embedding-3-small',
'input' => $text,
'dimensions' => 512,
]);
// Refresh the index less often during bulk indexing to reduce load
$client->indices()->putSettings([
'index' => 'articles',
'body' => [
'index' => ['refresh_interval' => '10s'],
],
]);Once you’ve fine-tuned your AI search feature, it’s time to deploy it to production. Make sure to test thoroughly and monitor performance closely after deployment.
With these final steps, you’ve successfully implemented a robust AI-powered search solution in Laravel. This feature will enhance user engagement and provide valuable insights into your application’s content.
Frequently Asked Questions
What are the benefits of implementing Elasticsearch and OpenAI in a Laravel application for search functionality?
Elasticsearch provides fast and scalable search capabilities, while OpenAI embeddings enable contextually aware and customizable search results.
How do I troubleshoot issues with slow query performance in my Elasticsearch index?
Check your index settings, mapping, and query syntax to ensure they are optimized for performance. Consider reindexing or adjusting the number of shards.
Is there an alternative approach to using OpenAI for search functionality in Laravel?
Yes, you can use hosted search engines like Algolia or Meilisearch, both of which work with Laravel Scout and offer their own AI-powered search features. Combining OpenAI embeddings with Elasticsearch gives you more control over models, scoring, and customization.
What is the purpose of the search_index database migration?
The migration creates the database table that stores your source content. The Elasticsearch index itself is created by the Elasticsearch client (in our search:index command), which reads the rows from this table and indexes them with their embeddings.
Why do I need to install the Elasticsearch client in addition to OpenAI?
The Elasticsearch client stores your documents and their embeddings and runs the actual search queries, while OpenAI generates the embeddings that make the search contextually aware.