Testing

Rotary features a concise unit testing implementation, imaginatively named ktest. This is used to implement tests for various subsystems, including the bootmem allocator and page frame allocator.

Currently, not all subsystems feature a test suite, however I intend to implement these in future.

Overview

All unit tests are currently located in the /test directory, and included at the end of their relevant subsystem’s source file. Including the tests in this manner allows them to access data structures and variables defined within the scope of the subsystem’s source files.

An individual unit test is defined by a ktest_unit_t struct. A macro is available to initialise a unit test struct, KTEST_UNIT(name, function).

A collection of unit tests is defined by a ktest_module_t struct. This is used to logically group unit tests, for example into “bootmem” and “palloc” modules.

Modules are defined using the KTEST_MODULE_DEFINE macro, which initialises a ktest_module_t with the provided parameters, and ensures that the struct is included in the .ktest executable section by the linker.

Modules are located in the .ktest section, allowing them to be dynamically discovered at runtime. By placing ktest_module_t structures in this section, the test framework can iterate through them without requiring explicit registration. This means adding a new module is as simple as defining a ktest_module_t struct — no manual invocation of a registration function is needed. The linker ensures all modules are placed in .ktest, making test discovery automatic.

Functionality

ktest offers four function hooks that can be used to configure state, or perform any other operations necessary pre and post execution of a test suite.

When registering your test module, you can provide these hooks, for example you may wish to provide:

  • example_pre_module()
  • example_post_module()
  • example_pre_test()
  • example_post_test()

These functions can be used to ensure that a certain state is maintained at the start of each unit test, and that the actions of one unit test will not affect another.

In the bootmem test suite, these hooks are used to ensure that the mem_regions array is cleared, and the PFN/region count set to zero before execution of each test. This allows each test to begin with a clean slate.

The assertion functions provided are as follows:

  • assert_equal(observed, expected) - Asserts whether the two values are equal.
  • assert_not_equal(observed, expected) - Asserts whether the two values are not equal.
  • assert_clear(buffer, size) - Verifies that a memory buffer is zeroed out.
  • assert_filled(buffer, size, value) - Checks if a buffer is filled with a specific value.
  • assert_bit_set(buffer, bit) - Ensures a specific bit in an integer is set.

If a test passes, no message is printed to the kernel log, as this may be overly verbose depending on the test count. If a test fails, a line is printed detailing the failure location, the expected value, and the actual observed value:

[kernel/test/ktest.c] [palloc-buddy-init] int ASSERTION FAILED! exp 1 != act 0 (file ./test/palloc.c line 101)

At the end of a module’s execution, a summary of each unit test and its pass/fail count is printed to the serial debug log:

[kernel/test/ktest.c] === Module Summary ===
[kernel/test/ktest.c] [ OK ] palloc-buddy-init (133143 passes)
[kernel/test/ktest.c] [ OK ] palloc-test-min-order (17 passes)
[kernel/test/ktest.c] [ OK ] palloc-test-max-order (17 passes)
[kernel/test/ktest.c] [ OK ] palloc-test-alloc-exhaust-all (526 passes)
[kernel/test/ktest.c] [ OK ] palloc-test-alloc-split-and-free (12 passes)
[kernel/test/ktest.c] [ OK ] palloc-test-free-null (1 passes)
[kernel/test/ktest.c] [ OK ] palloc-test-free-critical (3 passes)
[kernel/test/ktest.c] [ OK ] palloc-test-is-critical (2 passes)
[kernel/test/ktest.c] [ OK ] palloc-test-partial-block-free (16 passes)
[kernel/test/ktest.c] ======================

Example: Creating a New Test Suite

Below I’ve included a definition of an example test suite, that makes use of all of the functionality currently available in ktest. If this file were to be included in the /test directory as example.c, it would be automatically registered as an example module, and invokable using ktest_run_module('example'). Throughout the file, I’ve included some basic comments that explain the purpose of each section.

/*
 * test/example.c
 * Example Test Module
 */

#include <rotary/test/example.h>

/* ------------------------------------------------------------------------- */
/* Test Set-up and Clean-up                                                  */
/* ------------------------------------------------------------------------- */

/* Any actions you want to perform before the test module begins, for example
   configuration of initial state. This will only be run once, so any changes
   made by individual tests to data structures configured in this section will
   persist until the end of the module's execution. */
int32_t example_pre_module(ktest_module_t * module) {
    return E_SUCCESS;
}

/* ------------------------------------------------------------------------- */

/* Any actions you want to perform after the test module has finished its
   execution. This could be used to provide additional diagnostic logging,
   if desired. */
int32_t example_post_module(ktest_module_t * module) {
    return E_SUCCESS;
}

/* ------------------------------------------------------------------------- */

/* Any actions you want to perform before each individual unit test function
   is executed. This is the most appropriate place to ensure that data
   structure state is reset to a predictable value each time, and that any
   changes made by a unit test do not persist to the next function. */
int32_t example_pre_test(ktest_module_t * module) {
    return E_SUCCESS;
}

/* ------------------------------------------------------------------------- */

/* Any actions you want to perform after an individual unit test function has
   executed. */
int32_t example_post_test(ktest_module_t * module) {
    return E_SUCCESS;
}

/* ------------------------------------------------------------------------- */
/* Unit Tests                                                                */
/* ------------------------------------------------------------------------- */

void example_test_pass(ktest_unit_t * ktest) {
    assert_equal(1, 1);
}

/* ------------------------------------------------------------------------- */

void example_test_fail(ktest_unit_t * ktest) {
    assert_equal(1, 0);
}

/* ------------------------------------------------------------------------- */

void example_test_math(ktest_unit_t * ktest) {
    int a = 2, b = 3;
    assert_equal(a + b, 5);
    assert_not_equal(a * b, 4);
}

/* ------------------------------------------------------------------------- */
/* Test Registration                                                         */
/* ------------------------------------------------------------------------- */

/* You must add each of your defined functions as a `ktest_unit_t` using the
   `KTEST_UNIT()` initialisation function. */
static ktest_unit_t test_units[] = {
    KTEST_UNIT("example-pass", example_test_pass),
    KTEST_UNIT("example-fail", example_test_fail),
    KTEST_UNIT("example-math", example_test_math),
};

/* By invoking this macro, the module will be saved into the `.ktest`
   section of the final image, making it available enumeration and invocation
   at runtime. */
KTEST_MODULE_DEFINE("example", test_units,
                    example_pre_module,
                    example_post_module,
                    example_pre_test,
                    example_post_test);

/* ------------------------------------------------------------------------- */

Relevant Source