Laravel Queues: An Asynchronous Processing

This article aims to guide you through the process of executing jobs asynchronously using queues and workers in Laravel.

When it comes to web applications and softwares, speed is crucial for a satisfactory user experience. However, certain tasks require more time to complete, such as sending automated emails, importing large data from CSV or excel files or generating complex reports from extensive data sets. These kinds of tasks cannot be performed synchronously. The well-designed web applications utilize such asynchronous processing in the background, which can be performed using a capability provided by Laravel Queues.

Prerequisites:

This article aims to guide you through the process of executing jobs asynchronously using queues and workers in Laravel. Prior to diving in, please ensure that you have PHP, Composer and MySQL installed on your system, and that you are familiar with bootstrapping and initiating a new Laravel project.

The Concept:

Before delving into the code, I would like to describe the concept of asynchronous processing with a real-life analogy. Imagine yourself as the owner of a local grocery shop that accepts orders over the phone and delivers the requested items to customers’ doorsteps.

Now, assume that you receive a phone call from a customer and take a new order. You write this order on the piece of paper. You then proceed to the storeroom, locate the individual products, pack them into a box, and arrange for delivery. In this scenario, you cannot accept any new orders until you have dispatched the current one. This is called synchronous approach. It blocks your ability to handle other tasks until you complete the currently running process at hand.

However, instead of sticking to a synchronous model, you have the option to hire an additional worker for the storeroom. By doing so, you can accept a new order, jot it down on a piece of paper (referred to as a job), and place it in a queue. One of the storeroom workers will pick a job from the queue, assemble the order, and arrange for delivery. Afterwards, the worker will return to the queue, select the next job, and commence its processing. This is called an asynchronous approach. It  allows for a more efficient workflow. While employing extra workers may require additional resources, it enables you to handle more orders without impeding your overall productivity. This, in turn, leads to improved performance and heightened customer satisfaction.

The Basic Implementation:

Now, let’s see how to implement this asynchronous approach of a job, a worker, and a queue in Laravel. To start, create a new Laravel project somewhere on your computer. Open the project in your favourite IDE and update the code for the routes/web.php file as follows:

Route::get('/', function () {

    dispatch(function() {
        sleep(5);
        logger('job done!');
    });

    return view('welcome');
});

The dispatch() helper function in Laravel sends a new job to a queue. Here, a job is regular PHP code in the form of a closure or class. You can also use job classes instead of closures in above code.

To simulate a long-running task, I have added an explicit delay of five seconds using the sleep function in the job closure. Then, I used the logger() helper function to print the line “job done!” in the project’s laravel.log file located at storage/logs/laravel.log.

Now, start the project by running the following command:

php artisan serve

And open http://127.0.0.1:8000 in your browser. You will see the browser loading for five seconds, and then the welcome page will show up. If you check the log file at location storage/logs/laravel.log, you will see the following log entry:

[2021-08-23 14:37:50] local.DEBUG: job done!

This means that the job ran successfully after the five-second delay, but still it is not running asynchronously. It is still blocking the I/O operations as the browser was in a loading state for five seconds. It is because of the QUEUE_CONNECTION variable in the project’s .env file. This variable indicates the connection to be used in the queue service in Laravel. By default, it has been set to sync, which means that Laravel will process all the jobs synchronously.

Asynchronous Approach:

For asynchronous job-processing, you will have to use a different connection. Laravel has provided a pre-configured list of connections inside the config/queue.php file as follows:

'connections' => [

    'sync' => [
        'driver' => 'sync',
    ],

    'database' => [
        'driver' => 'database',
        'table' => 'jobs',
        'queue' => 'default',
        'retry_after' => 90,
        'after_commit' => false,
    ],

    'beanstalkd' => [
        'driver' => 'beanstalkd',
        'host' => 'localhost',
        'queue' => 'default',
        'retry_after' => 90,
        'block_for' => 0,
        'after_commit' => false,
    ],

    'sqs' => [
        'driver' => 'sqs',
        'key' => env('AWS_ACCESS_KEY_ID'),
        'secret' => env('AWS_SECRET_ACCESS_KEY'),
        'prefix' => env('SQS_PREFIX', 'https://sqs.us-east-1.amazonaws.com/your-account-id'),
        'queue' => env('SQS_QUEUE', 'default'),
        'suffix' => env('SQS_SUFFIX'),
        'region' => env('AWS_DEFAULT_REGION', 'us-east-1'),
        'after_commit' => false,
    ],

    'redis' => [
        'driver' => 'redis',
        'connection' => 'default',
        'queue' => env('REDIS_QUEUE', 'default'),
        'retry_after' => 90,
        'block_for' => null,
        'after_commit' => false,
    ],
],

You can use any connection except sync,which is used for synchronous job-processing. For this example, we will use the database connection. To update the connection, open the project’s .env file and change the value of QUEUE_CONNECTION to database and save the file.

But, to work with database connection, we have to create a table to maintain our asynchronous jobs. To create the migration for this table, run the following command:

php artisan queue:table

It will generate the migration script for creating the jobs table. Open the newly generated migration script, it will have the following content:

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
    /**
     * Run the migrations.
     */
    public function up(): void
    {
        Schema::create('jobs', function (Blueprint $table) {
            $table->bigIncrements('id');
            $table->string('queue')->index();
            $table->longText('payload');
            $table->unsignedTinyInteger('attempts');
            $table->unsignedInteger('reserved_at')->nullable();
            $table->unsignedInteger('available_at');
            $table->unsignedInteger('created_at');
        });
    }

    /**
     * Reverse the migrations.
     */
    public function down(): void
    {
        Schema::dropIfExists('jobs');
    }
};

Now, run the migration command as follows to create the jobs table in the database:

php artisan migrate

This table will hold all the queued jobs until a worker processes them. Before starting the laravel, clear the project’s log file located at storage/logs/laravel.log and run the following command:

php artisan serve

And open http://127.0.0.1:8000 in your browser. This time, the page will render almost immediately. After waiting for five seconds, if you check the storage/logs/laravel.log file, you’ll find it empty. Because, the above setup will only add the job into the jobs table. But, execution of this job is still pending.

If you look at the jobs table in your database, you’ll see that the framework has pushed a new job to the queue. To execute this job, we have to run the Laravel queue worker process.

Laravel Queue Worker:

A queue worker is a regular process that runs in the background and polls the queue for unprocessed jobs. To start a new worker, execute the following command inside your project directory:

php artisan queue:work

The worker will start and begin processing the unprocessed job immediately. Now, check the storage/logs/laravel.log file. It will display the following logs: 

[2021-08-23 15:30:34][1] Processing: Closure (web.php:18)
[2021-08-23 15:30:39][1] Processed:  Closure (web.php:18)

Note: To process jobs automatically in the future, you’ll have to keep the worker process running.

And that concludes this article. I have covered the basic concepts of how synchronous and asynchronous approach are working and how you can use Laravel Queues to run asynchronous jobs. There are numerous other aspects of queue management that you should familiarize yourself with, such as job uniqueness, handling race conditions, and implementing throttling. It is advisable to begin incorporating queues into your applications and learn through hands-on experience. Additionally, make sure to thoroughly explore the official documentation on Laravel Queues. If you have any questions or concerns about any of the concepts presented in this article, please feel free to reach out to me on LinkedIn, or GitHub. Until the next article, stay safe and continue your learning journey.

Laravel order by relation column with example

Some examples that demonstrate the usage of Laravel’s orderBy method with relationship columns, which can be applied in Laravel versions 5, 6, 7, 8, 9, and 10.

In this post, we focus on examples to use Laravel order by relation column. There are many different ways to achieve it.

Below are some examples that demonstrate the usage of Laravel‘s orderBy method with relationship columns. These examples can be applied in Laravel versions 5, 6, 7, 8, 9, and 10.

Example 1: Ordering by a BelongsTo Relationship Column

Consider the scenario where you have a "User" model that belongs to a "Role" model. You can use the orderBy method to sort the users based on the role name in ascending order.

$users = User::with(['role' => function ($q) {
    $q->orderBy('name');
}])->get();

You can use the same code for descending order by adding 'desc' argument to orderBy method as follows,

$users = User::with(['role' => function ($q) {
    $q->orderBy('name', 'desc');
}])->get();

For above example to work properly, you have to define belongsTo relationship of "Role" model inside the "User" model as follows,

public function role(): BelongsTo
{
    return $this->belongsTo(Role::class, 'role_id', 'id');
}

Example 2: Using inner join with relation table

For above scenario, where “User” model that belongs to “Role” model, you can use join method to perform user sorting based on role name as follows,

$users = User::select('*')
                 ->join('roles', 'users.role_id', '=', 'roles.id')
                 ->orderBy('roles.name', 'asc');

As above example, you need to replace the second argument of orderBy method to 'desc' for sort records in descending order. There is no need of any relationship required for this query.

Example 3: Using sortBy() and sortByDesc() methods

You can use sortBy() and sortByDesc() methods to order the records for the same scenario as follows,

$users = User::get()->sortBy(function($query){
    return $query->role->name;
})->all();

In above example, it first get the users collection from the database and then sort them by the provided relation column. For these methods, you have to define belongsTo relationship in "User" model.

You can use sortByDesc() method same as above.

Example 4: Using subquery and whereColumn method

You can also use subqueries to sort records. For the above scenario, you can use subqueries as follows,

$users = User::select('*')
    ->orderBy(Role::select('name')
        ->whereColumn('roles.id', 'users.role_id')
    );

In above example, we have we used subquery from roles table using whereColumn method to get the name of the role of each row of users table. After getting the role name, we used orderBy method to achieve the sorted records.

For sorting records in descending order, you can use the above example with orderByDesc method.

You can use this example, without defining belongsTo relationship in model.

These are some useful examples to order records based on relationship model in Laravel.

Using Laravel to upload a file

Uploading a file in any programming is a challenge. So, we focus on uploading a file and some validations to use with file upload using Laravel.

Uploading a file in any programming is a challenge. In this post, we focus on uploading a file and some validations to use with file upload using Laravel.

To upload a file using Laravel, you can follow these steps:

Create a new form in your Laravel view with an input field for the file:

<form method="POST" action="{{ route('file.upload') }}" enctype="multipart/form-data">
    @csrf

    <input type="file" name="file">

    <button type="submit">Upload</button>
</form>

In above code, we have added file.upload route as an action of the form. So, we need to define this route in routes/web.php file. This route should point to the controller method that will handle the file upload.

The following code will define a new route in your routes/web.php file:

Route::post('/file/upload', [App\Http\Controllers\FileController::class, 'upload'])->name('file.upload');

Above code has defined a route, which points to the upload method of the FileController. So, create a new controller FileController and method upload inside that to handle the file upload as follows:

class FileController
{
    public function upload(Request $request)
    {
        // Validate the uploaded file
        $request->validate([
            'file' => 'required|file|max:1024', // limit file size to 1 MB
        ]);

        // Store the uploaded file in the storage/app/public directory
        $path = $request->file('file')->store('public');

        // Generate a URL for the uploaded file
        $url = Storage::url($path);

        // Redirect back with a success message
        return back()->with('success', 'File uploaded successfully: ' . $url);
    }
}

In the upload() method, we first validate that the uploaded file meets our requirements. So, we have added some validations for our file. These validations are as follows:

required:

The field under this validation must be present in the input data and must not empty. A field is “empty” if it meets one of the following criteria:

  • The value is null.
  • The value is an empty string.
  • The value is an empty array or empty Countable object.
  • The value is an uploaded file with no path.

file:

The field under this validation must be a successfully uploaded file.

max:1024:

The field under this validation must be less than or equal to a 1024 bytes. Here, 1024 is value of file size. You can change it according to your requirements.

We then use the store() method on the uploaded file to store it in the storage/app/public directory. This directory is publicly accessible, so we can generate a URL for the file using the url() method on the Storage facade. Finally, we redirect back to the form with a success message that includes the URL of the uploaded file.

You can now test the file upload functionality by navigating to the form and selecting a file to upload. If the file meets the validation requirements, it will be uploaded and a success message will be displayed. You can then access the uploaded file at the generated URL.

Solved – error while loading shared libraries: libpangox-1.0.so.0: Anydesk on Ubuntu 22.04 LTS

After successfully upgrading from Ubuntu 20.04 LTS to 22.04 LTS, most of the applications are working perfectly. But, some of the applications behave unusually. Such as, I tried to run the Anydesk application, but it doesn’t launch/start. when I checked the status service, the AnyDesk service was failed and the reason for failing is mentioned in the below error message,

anydesk: error while loading shared libraries: libpangox-1.0.so.0: cannot open shared object file: No such file or directory

libpangox-1.0.so.0 is a library used for text layout and rendering the text. Most of the work on Pango-1.0 was done using the GTK+ widget toolkit as a test platform.

So, I ran the command to install it using the apt as follows,

apt install libpangox-1.0-0

But, it gives me the following error,

Package libpangox-1.0-0 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

So, from that we can understand that, libpangox-1.0.so.0 can’t be installed from apt or apt-get. We need to install it manually.

Use the following steps to install libpangox-1.0.so.0 manually,

Step 1. Download the libpangox-1.0 package

wget http://ftp.us.debian.org/debian/pool/main/p/pangox-compat/libpangox-1.0-0_0.0.2-5.1_amd64.deb

Step2: Install the package using apt

sudo apt install ./libpangox-1.0-0_0.0.2-5.1_amd64.deb

Step3: Restart the AnyDesk service

sudo service anydesk restart

After these steps, if you check the AnyDesk service status, it will show active (running).

Error Handling in PHP (Part 2)

Now that we know, how to log errors in any system developed in PHP, we can move to our next section for keeping track of these logged errors. If you haven’t read how to log errors, read part 1 of error handling in PHP.

To keep track of these logged errors, we need to create a script to read those log files in a systematic way. Refer to the below code to read log files,

public function errorLogs($filePath = 'error.log') {

        $fileContent = file($filePath);

        $errorsArray = array();
        if(sizeof($fileContent) == 0) {
            return false;
        }

        foreach($fileContent as $row) {
            $errors = explode(":  ", $row);

            if(empty($errors[1])) continue;
            $errorsArray[] = $errors;
        }

        return array_reverse($errorsArray, true);
}

Explanation:

$fileContent = file($filePath);

This line of code will read the file line by line from the provided file path.

if(sizeof($fileContent) == 0) {
    return false;
}

After reading the file, if the size of the file content is 0 then, the function will return false. So, the purpose of this function is to stop the execution of the function if the provided file is empty and returns false.

foreach($fileContent as $row) {
      $errors = explode(":  ", $row);

      if(empty($errors[1])) continue;
      $errorsArray[] = $errors;
}

This part of the function will loop through the log contents row by row. For each row, it will explode the line with ‘:’ to separate the date and actual error details.

If the error details are empty for any row, it will skip that row. Otherwise, it will collect the errors in another array.

return array_reverse($errorsArray, true);

The last line of the function will reverse the error data and returned the reversed result. So, that we can see the latest errors first.

This way we can create a simple function to display the list of errors in a tabular format from the error log files we generated for each of the modules in the application system.