Guide for (Parallel) testing

[BertKooij]

Before I was using this package I had some trouble setting up testing for a multitenant application. Luckily the structure of this package made this a lot easier.

The problem I was having before is that with each test I was actually testing the mulitenant logic as well, which resulted in very slow tests. And whenever something was broken a lot of tests were failing, which made it a struggle to find out what the cause was.

The application I am working has grown quite a bit over the last few years, and the need for adding automated tests is already there for some time. Testing is actually already part of the CI steps for a few years, but I have to be honest and admit that there were only some basic Unit tests running. This year I finally took the time to improve the testing and made it possible to add tests for new features and progressively adding tests for the existing features.

The setup I currently have is:

• Having a testing database on a SQL server container (Laravel Sail + MSSQL Docker container);
• Making sure that the migrations for the landlord and tenant database are up to date before running the tests;
• Making sure there is a tenant added in the landlord database before running the tests;
• Tests that run for a tenant will make the first tenant current in the setUp method and will add the host to all the requests being tested. This will then make sure the tenant is found based on the host within the DomainTenantFinder.

Currently I have added 50 tests, which are testing some API endpoints. The average time of a test is roughly 1 second. Which makes running a full test sweep take around 50 seconds. Which is quite a shock because these tests cover only a very small percentage of the full application. I can imagine that we might end up with at least a 1000 tests, which will roughly take 15 minutes to run...

Example of a test:

    /** @test */
    public function can_add_a_calendar()
    {
        $user = $this->userWithPermissions(['manage.calendars']); // Will create a user from a factory and gives the user certain permissions

        $this
            ->actingAs($user)
            ->post('/api/v2/calendars', [
                'name' => 'Test calendar'
            ])
            ->assertStatus(201)
            ->assertJsonFragment([
                'name' => 'Test calendar'
            ]);

        $this->assertDatabaseHas('calendars', [
            'name' => 'Test calendar'
        ], 'tenant');
    }

Parallel testing will improve the execution time. With 4 parallel processes it would take 4 minutes (give or take). But until now I do not have a lot of luck with setting up parallel testing in combination with this package. There is already something going on about this in issue #97. After a little research and debugging I came across this PR laravel/framework#37264 which seems to be related to my troubles. The master database will be created for each parallel process (for example: landlord_test_1, landlord_test_2) which you can also seed after the creation, but the tenant database is not being created and seeded. Which will result in locks in combination with DatabaseTransactions.

So to start a discussion:

• Is anybody else having trouble with setting up a solid test suite for a multitenant application?
• Has anybody got parallel testing working with multitenancy?
• Is anybody struggling with the performance for testing with migrating tenant migrations and seeding the database?

It would be great if we can discuss how parallel testing could be used for multiple database connections in combination with his package. And as a bonus this discussion might help others with setting up automated testing.