History log of /freebsd-current/tests/conftest.py
Revision Date Author Comments
# 3e5d0784 14-Apr-2023 Alexander V. Chernikov <melifaro@FreeBSD.org>

Testing: add framework for the kernel unit tests.

This changes intends to reduce the bar to the kernel unit-testing by
introducing a new kernel-testing framework ("ktest") based on Netlink,
loadable test modules and python test suite integration.

This framework provides the following features:
* Integration to the FreeBSD test suite
* Automatic test discovery
* Automatic test module loading
* Minimal boiler-plate code in both kernel and userland
* Passing any metadata to the test
* Convenient environment pre-setup using python testing framework
* Streaming messages from the kernel to the userland
* Running tests in the dedicated taskqueues
* Skipping or parametrizing tests

Differential Revision: https://reviews.freebsd.org/D39385
MFC after: 2 weeks


# d9af4219 01-Apr-2023 Alexander V. Chernikov <melifaro@FreeBSD.org>

tests: refactor atf_python a bit

* Move more logic from conftest.py to the actual atf_pytest handler
* Move nodeid_to_method_name() to the utils.py so it can be shared

MFC after: 2 weeks


# 6332ef89 09-Feb-2023 Alexander V. Chernikov <melifaro@FreeBSD.org>

testing: handling non-root users with VNETs in pytest-based tests.

Currently isolation and resource requirements are handled directly
by the kyua runner, based on the requirements specified by the test.
It works well for simple tests, but may cause discrepancy with tests
doing complex pre-setups. For example, all tests that perform
VNET setups require root access to properly function.

This change adds additional handling of the "require_user" property
within the python testing framework. Specifically, it requests
root access if the test class signals its root requirements and
drops privileges to the desired user after performing the pre-setup.

Differential Revision: https://reviews.freebsd.org/D37923
MFC after: 2 weeks


# eaeebfcb 28-Dec-2022 Jose Luis Duran <jlduran@gmail.com>

pytest: silence deprecation warning in the pytest atf wrapper

Convert `pytest.mark` decorator to the `pytest.hookimpl()` one,
as suggested by the
https://docs.pytest.org/en/latest/deprecations.html#configuring-hook-specs-impls-using-markers

Differential Revision: https://reviews.freebsd.org/D37884
MFC after: 2 weeks


# 513ce835 28-Jun-2022 Alexander V. Chernikov <melifaro@FreeBSD.org>

testing: pass ATF vars to pytest via env instead of arguments.

This change is a continuation of 9c42645a1e4d workaround.
Apparently pytest argument parser is not happy when parsing values
with spaces or just more than one --atf-var argument.
Switch wrapper to send these kv pairs as env variables. Specifically,
use _ATF_VAR_key=value format to distinguish from the other vars.

Add the `atf_vars` fixture returning all passed kv pairs as a dict.

Reviewed by: lwhsu
Differential Revision: https://reviews.freebsd.org/D35625
MFC after: 2 weeks


# 8eb2bee6 25-Jun-2022 Alexander V. Chernikov <melifaro@FreeBSD.org>

testing: Add basic atf support to pytest.

Implementation consists of the pytest plugin implementing ATF format and
a simple C++ wrapper, which reorders the provided arguments from ATF format
to the format understandable by pytest. Each test has this wrapper specified
after the shebang. When kyua executes the test, wrapper calls pytest, which
loads atf plugin, does the work and returns the result. Additionally, a
separate python "package", `/usr/tests/atf_python` has been added to collect
code that may be useful across different tests.

Current limitations:
* Opaque metadata passing via X-Name properties. Require some fixtures to write
* `-s srcdir` parameter passed by the runner is ignored.
* No `atf-c-api(3)` or similar - relying on pytest framework & existing python libraries
* No support for `atf_tc_<get|has>_config_var()` & `atf_tc_set_md_var()`.
Can be probably implemented with env variables & autoload fixtures

Differential Revision: https://reviews.freebsd.org/D31084
Reviewed by: kp, ngie