Software Testing – Integration, Unit and Smoke Tests
Software development requires testing.
There are two main approaches to test application: manuall testing and automatic.
Manual option requires browser and person who will go through application workflow and sees the end results. It’s valuable work, but requires much effort from human side. On the other side are the automatic tests. It’s a big topic with many branches, but gives us incredible power for development of huge, readable, reliable applications.
Nowadays testing became internal part of every application. There are many aspects which should include tests. As an example – we as developers, are responsible to take care how our software is crated. If we are using external code (which is common since composer started to be popular tool), we are responsible to check if we are implementing something, which work as we expect. Tests in the code are something which helps in that. If code doesn’t contain any tests, we should be afraid of using such part of code. In the enterprise world, the tests existent is obvious either for business or client.
In this article we will focus on one of the main approach in testing which is TDD methodology and a toll called PHPUnit. For that – Laravel provide us out of the box a prepared testing environment, which is easy to start working with. In the root directory we can find a file: phpunit.xml, which allows us to customize the testing settings to our needs. Also all of the tests files and folders should be placed inside of the tests folder.
Smoke Tests
Tests Pyramid shows the layers of tests relations. On the top are smoke tests, which are a part of tests group called confidence testing. Those tests build the simple base coverage for application. Smoke tests are a subset of test cases that cover the most important functionality of a component or system. There is the first line where we see the application troubleshoots. It helps in uncovering problems early and used to aid assessment if main functions of the software appear to work correctly. As an example, lets say we have and enpoint of API, which should return some objects collection. In such case, we should write smoke tests, which will check if that enpoint is still working by just checking its response status ( 200). If we receive other status (e.g. 500) we can be sure that something is not working and we should put some attention to that part.
|
1 2 3 4 5 6 7 8 9 |
/** @test */ public function products_show_list() { $response = $this->get(route('products')); $response->isOk(); $response->assertViewIs('products'); $response->assertSee('Products'); } |
Above you can find an of simple smoke test, which take less than 5 lines. It is sending a request to specific enpoint and save the response in $response variable. Then on the next 3 lines we are checking:
- isOk –correctness of response status code
- assertViewIs –correctness of view rendering
- assertSee –correctness of fill out the template by text
We see that making the smoke tests is easy and fast to run.
It is easy to pick the highest advantages of smoke tests. It covers most of the major functions of the software (but none of them in depth) and shows that our build is stable enough to proceed with further testing.
The Framework is rapidly growing and often upgrading is required to keep up today. Every time after upgrading, the passed tests give us confidence that the application works fine.
I’m pretty sure that you work with legacy code time to time. This code usually hasn’t got any tests. If you begin since writing the smoke tests, you will get to know how the application works. Future refactoring will be also easier without fear.
The main value of smoke tests is the speed of writing them and also speed of execution. The execution time is important if you have a huge software, which contain many integration and unit tests. During continuous integration, we can run first the smoke tests. If we see that something is going wrong – then we don’t run other tests (since there is no sense for that). The other tests are fired only if the smoke tests will be ended successfully. This allows us to save some time, because the integration and unit tests execute much longer.
Integration Tests
Testing performed to expose defects in the interfaces and in the interactions between integrated components or systems.
We want to test the integrity without entering in implementation details. We know what functionally is expected on output of a component and we verify it by putting the inputs and observe the result. If everything went well the output results will be equals as expected. What can went wrong? The tests shows incorrect or missing functions, interface errors which helps keeping integrity with other components. It exposes errors in data structures or external database access. This approach allows to see how behaves the whole application and eventually, we would detect the performance errors like application lags, time of waiting for response, consuming resources. During following test, the application state evaluates, and it could detect initialization and termination errors.
All mentioning aspects allows to think about integration tests as the group of small unit parts cooperating together. Lets take the ballpoint pen as an analogy of that process:
When two or more units are ready, they are assembled and Integration Testing is performed. For example, whether the cap fits into the body or not.
Modern application are equipped in database. No matter, is it MySQL, SQLite or NoSQL. We have in mind to write tests. Laravel doesn’t leave us in that area and it provides faking datasets by Factories usage.
Below is the factory definition snippet example:
|
1 2 3 4 5 6 7 8 |
$factory->define(App\Models\User::class, function(Faker $faker) { return [ 'first_name' => $faker->name, 'last_name' => $faker->lastName, 'email' => $faker->unique()->safeEmail, 'password' => bcrypt('secret') ]; }); |
By the factory helper usage, we create a new fake instance of given model. We have two options where to keep the data: in database by using the “create” method or in memory using the “make” method:
|
1 2 3 4 5 6 7 8 9 10 |
$this->user = factory(User::class)->make([ ]); $this->presisting_user = factory(User::class)->create(); $this->presisting_users = factory(User::class, 3)->create() ->each(function($u) { $u->products()->save(factory(\App\Models\Product::class)->make()); }); |
The returned object is fully acceptable object instance. It will be used later in application flow.
Working with database required tools. The laravel provides appropriate methods making the work with database easy and comfortable. Let’s extract the basic three:
- assertDatabaseHas($database_table, $searching_details) – assert that a table in database contains the given data
- assertDatabaseMissing ($database_table, searching_details) – assert that a table in database does not contain the given data
- assertSoftDeleted($database_table, $searching_details) – assert that the given record has been soft deleted
Let’s look at the sample test below:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
/** @test */ public function store_user_stored_in_DB() { $data = factory(User::class)->make()->toArray(); $initial_data = $data + [ 'password' => 'passA1#', 'password_confirmation' => 'passA1#' ]; $json = $this->json('post', route('users.store'), $initial_data) ->assertStatus(201); $this->assertDatabaseHas('users', $data); } |
It’s checking if the user which we created in memory (using Factory) was stored in database by sending request to application json API. Let’s analyze step by step.
Our dataset is initiated user object which we put into an array in data variable. Next we add the password require fields validation step. Having the input data prepared, we send the request to API and we check the correctness of response status code (201 according HTTP status codes). Ending test we assert the database state with expected state. If they are the same, it means, that the user was stored in db.
Above we mentioned about the application state, having in mind that a database should be consistent with an application. Laravel provides useful traits:
- Illuminate\Foundation\Testing\DatabaseTransactions
- Illuminate\Foundation\Testing\RefreshDatabase(since Laravel 5.5)
which help to work with database. They care about keeping all database changes during one test in transaction, which is rollbacking after test is finished. It provide consistent of the database with the starting state. The second trait allows to do the database migrations before test are fired.
Despite the Factories impact for early dataset and transaction mechanism, it’s recommend to separate the database between testing environment and simultaneous regular application. The easiest way is to define the dedicated configuration instance in app/config/database.php and set the accordingly environment variables as see below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
'db_testing' => [ 'driver' => 'mysql', 'host' => env('DB_HOST_TESTING', '127.0.0.1'), 'port' => env('DB_PORT', '3306'), 'database' => env('DB_DATABASE_TESTING', 'forge'), 'username' => env('DB_USERNAME', 'forge'), 'password' => env('DB_PASSWORD', ''), 'unix_socket' => env('DB_SOCKET', ''), 'charset' => 'utf8mb4', 'collation' => 'utf8mb4_unicode_ci', 'prefix' => '', 'strict' => true, 'engine' => null, ], |
Laravel has phpunit.xml configuration in project root directory. There is XML schema where we set database environment variable using through tests.
<env name=”DB_CONNECTION” value=”db_testing”/>
Summarizing, we can test components in many cases, how them influence expected results. Below list consists of the most common application modules:
- validation
- authentication (Laravel guards, Web, Api – Oauth)
- authorizations (Laravel Policies)
- database Storing
- pagination
- filters
- response (HTTP Code + message)
Unit Tests
The purpose of unit testing is to validate that each unit of the software performs as designed, where under the unit we understand the smallest testable part of code. The smallest unit is a method, which may belong to a class. It usually has one or a few inputs and usually a single output.
In unit tests we use the Mocks. Mock objects simulate the behavior of real objects (or we should write: an instance of pointed class). They are commonly utilized to offer test isolation, to stand in for objects which do not yet exist, or to allow for the exploratory design of class APIs without requiring actual implementation up front.
There are couple useful mocks frameworks:
- PHPUnit
- Phony
- Prophecy
- Mockery
We focus on Mockery. The semantic of mocking given class is readable and intuitive during building.
On the beginning the mocker is built.
We choose mocking method and set an expected number of occurrences of it during test flow and expected returned result. Next, the mocking object we set as injected instance for given class by binding our mock. Finest, we ask container for service instance where, the container injects our mock in. The assertion passes in predictable way and test went fine.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
/** @test */ public function order_service() { // Create Mock $payment_service = \Mockery::mock(PaymentService::class); // Generated partial test doubles $payment_service->shouldReceive('paid') ->once() ->andReturn(true); // Binding Instance $this->app->instance(PaymentService::class, $payment_service); // Create Order using Container $order = $this->app->make(Order::class); // Assertion $this->assertSame(Order::SENT, $order->send()); } |
We can look in the sample implementation of Order class and mocking Payment Service.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
class Order { const SENT = 'order sent'; const NEW = 'new order'; private $status = self::NEW; private $send; /** * @var PaymentService */ private $payment_service; /** * Order constructor. * @param PaymentService $payment_service */ public function __construct(PaymentService $payment_service) { $this->payment_service = $payment_service; } public function send() { if ($this->payment_service->paid()) { $this->send = true; $this->status = self::SENT; } return $this->status; } } |
The Order instance looks for injection of payment service instance. This is called Depedency Injection and it makes code fully testable through replacing services by prepared mocks. It makes fully outside isolated environment.
|
1 2 3 4 5 6 7 |
class PaymentService { public function paind() { // } } |
We leave empty paid method in payment service implementation, because it’s mocked before.
Laravel has built-in well-known set of services called Facades. Laravel as a modern framework support the tools for helping build the test coverage of application. That way, the facades share the mocking options. The mocking steps are similar to servicing by Mockery library. We can use analogy methods like shouldReceive, once, andReturn. Below, there is a snippet, where we see the sample test which contains Lang Facade mock.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
/** @test */ public function order_service() { // Create mock \Lang::shouldReceive('trans') ->with('laravel') ->once() ->andReturn('master'); // Assertion $this->assertSame('Best Framework', (new Dictionary())('laravel')); } |
After using the mocking method, each time, when Laravel’s container meets the call for trans method on Lang facade, it uses then prepared mock methods.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
class Dictionary { function __invoke($word) { $lang = \Lang::trans($word); if ($lang === 'master') { return 'Best Framework'; } return 'all others'; } } |
Unit Testing is the lowest level of software testing which test the application logic.
Let’s see what the advantages from the unit tests approach are.
First and the most important is, the unit tests increases confidence in writing/changing/maintaining codebase. The simple testing unit means short and not complicate tests which implicates easy debugging. If we make mistake on arguments list or returned values, we will find bug immediately. We fix bug and the patch exists forever. Why? Every attempt of removing the patch breaking the test. In result the code became reliable and the future development is faster and cheaper because the cost of fixing a defect, detected during unit testing, is lesser in comparison to that of defects detected at higher levels. There are the cases where the application evolves or became have changes in specification. Then we need to rewrite the unit tests or move to another place – but with well testable application, the predictable application architecture follows, where putting changes is easy and doesn’t blow up the whole application.
Author: Kamil Piętka, backend programmer at Devpark Laravel Team
