Skip to content

Commit 9edb72f

Browse files
committed
feat(lftest): add attach_tests() helper for accurate test discovery
The prevailing pattern in monitoring-plugins unit tests has been class TestCheck(unittest.TestCase): check = '../my-plugin' def test(self): for t in TESTS: with self.subTest(id=t['id']): lib.lftest.run(self, self.check, t) That works in the sense that every testcase still runs, but unittest counts the surrounding `test` method as **one** test regardless of how many fixtures the TESTS list contains. Running `./run` therefore reports `Ran 1 test in ...s` even when the file exercises 24, 56 or more scenarios, and `./run -v` is unhelpful because it only lists the single method name. `attach_tests(test_class, tests)` walks the TESTS list once at import time and materialises one real `test_<sanitised id>` method on the class. From unittest's point of view there are now N individual tests with proper names, the test count is accurate and verbose output lists every scenario. The new pattern is one line: class TestCheck(unittest.TestCase): check = '../my-plugin' lib.lftest.attach_tests(TestCheck, TESTS)
1 parent efec109 commit 9edb72f

File tree

1 file changed

+68
-1
lines changed

1 file changed

+68
-1
lines changed

lftest.py

Lines changed: 68 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,11 +12,12 @@
1212

1313
import contextlib
1414
import os
15+
import re
1516

1617
from . import base, disk, shell
1718

1819
__author__ = 'Linuxfabrik GmbH, Zurich/Switzerland'
19-
__version__ = '2026041301'
20+
__version__ = '2026041302'
2021

2122

2223
def run(test_instance, plugin, testcase):
@@ -94,6 +95,72 @@ def run(test_instance, plugin, testcase):
9495
test_instance.assertRegex(stdout, testcase['assert-regex'])
9596

9697

98+
def attach_tests(test_class, tests, plugin_attr='check'):
99+
"""Dynamically attach one ``test_<id>`` method per testcase to a
100+
``unittest.TestCase`` subclass, so that every entry in the TESTS
101+
list shows up as an individual test in the unittest discovery
102+
output instead of being collapsed into a single ``test`` method
103+
with sub-tests.
104+
105+
### Why
106+
The naive approach is::
107+
108+
class TestCheck(unittest.TestCase):
109+
def test(self):
110+
for t in TESTS:
111+
with self.subTest(id=t['id']):
112+
lib.lftest.run(self, self.check, t)
113+
114+
That works, but unittest counts the whole loop as **one** test, so
115+
the user sees ``Ran 1 test`` regardless of how many fixtures the
116+
file actually exercises. Failures still surface (sub-tests print
117+
their `id`), but the test count is misleading and `./run -v` does
118+
not list each scenario. ``attach_tests()`` materialises one real
119+
test method per testcase so the count is accurate and verbose
120+
output names every scenario.
121+
122+
### Parameters
123+
- **test_class** (`type`): a ``unittest.TestCase`` subclass with a
124+
``check`` (or other ``plugin_attr``-named) attribute pointing at
125+
the plugin executable.
126+
- **tests** (`list[dict]`): a TESTS list of testcase dicts, each
127+
shaped as ``run()`` expects, with a unique ``id`` field.
128+
- **plugin_attr** (`str`, optional): the attribute name on
129+
``test_class`` that holds the plugin path. Defaults to
130+
``'check'``.
131+
132+
### Example
133+
>>> class TestCheck(unittest.TestCase):
134+
... check = '../my-plugin'
135+
...
136+
>>> attach_tests(TestCheck, TESTS)
137+
>>>
138+
>>> if __name__ == '__main__':
139+
... unittest.main()
140+
141+
The resulting class has a ``test_<sanitised id>`` method per
142+
entry in ``TESTS``. Running ``./run -v`` then lists every test
143+
by name and ``./run`` reports the real test count.
144+
"""
145+
seen = set()
146+
for testcase in tests:
147+
raw_id = testcase['id']
148+
method_name = 'test_' + re.sub(r'\W+', '_', raw_id).strip('_')
149+
if method_name in seen:
150+
raise ValueError(
151+
f'attach_tests: duplicate test id "{raw_id}" '
152+
f'maps to method name "{method_name}"'
153+
)
154+
seen.add(method_name)
155+
156+
def _make(captured_testcase):
157+
def _method(self):
158+
run(self, getattr(self, plugin_attr), captured_testcase)
159+
return _method
160+
161+
setattr(test_class, method_name, _make(testcase))
162+
163+
97164
@contextlib.contextmanager
98165
def run_container(
99166
image,

0 commit comments

Comments
 (0)