Spektrum is a Python testing framework inspired from RSpec and Jasmine. The library was created out of a desire to have a relatively flexible Python testing framework that adopted a more code-centric approach to BDD.
Spektrum is open-source and is available on GitHub. We love contributions!
This project is spec-driven, and OpenSpec is its only planning system. Anything beyond a
trivial fix starts as a change under openspec/changes/ — a proposal, spec deltas, a design
and tasks — before the code is written, and openspec/specs/ holds the agreed requirements
the library is held to. Read the relevant spec before changing behaviour, and delta it rather
than restating it.
The CLI is a Node package, separate from the Python toolchain:
npm install -g @fission-ai/openspec # >= 1.12.0 openspec list # in-flight changes openspec list --specs # the requirement baseline openspec validate --all --strict # what CI runs
Two checks run on every pull request, and both fail the build:
- validate runs
openspec validate --all --strict. A spec or change that does not parse is broken rather than untidy — it is invisible toopenspec list, it cannot be archived, and it has quietly stopped describing anything. - change-document requires the pull request either to include a spec file — a
spec.mdunderopenspec/changes/<change>/specs/or underopenspec/specs/— or to state why one is not appropriate.
A proposal, design or task list on its own is not a spec file: a proposal states motivation, a spec states the contract. There is no exemption by path, so a documentation-only pull request is covered like any other. To state an exception, add a line to the pull request description:
sdd-exception: dependency bump, no behaviour change
The reason is required — a bare sdd-exception: fails — and the job echoes it into its log,
so the justification survives in the build record.
Both checks are required, so a red one holds the merge. The exception line is the sanctioned override and it is not a weakness in that: it is what keeps blocking proportionate. Without a required check, a pull request can be merged having done neither thing — no spec file and no stated reason — leaving a red check nobody answers for. With one, it can be merged once one of the two is done, and the relief is a single sentence that needs nobody's approval and is itself the record of why the rule was set aside.
Travis owns the tests and the lint; these two own only the openspec/ artifacts. Red here is
paperwork; red on Travis is code.
Install from PyPI:
pip install spektrum
Tests are plain classes. A Spec subclass groups cases: every public method that is
not a lifecycle hook (before_all, before_each, after_each, after_all) and
not underscore-prefixed is a case. Nested Spec subclasses become child specs:
from spektrum import Spec, expect
class Addition(Spec):
def before_each(self):
self.total = 1 + 1
def sums_two_numbers(self):
expect(self.total).to.equal(2)
Run them by pointing the CLI at a path:
spektrum -s path/to/specs
Spektrum looks for a spec directory in the working directory when -s is omitted.
Add --show-all-expects to print a line per assertion rather than per case, and
-c N to run with concurrency.
While you iterate on one spec, narrow the run rather than sitting through the whole suite.
-p selects by module path -- either a bare module or a class inside it:
spektrum -s spec -p checkout # every spec in checkout.py spektrum -s spec -p checkout.PaymentFlow # one class and its children
Prefix the value with re: to match class paths by regular expression. The pattern is
anchored at the end, so it must describe the tail of the path -- a pattern that merely
appears in the middle matches nothing, with no error to tell you so:
spektrum -s spec -p "re:checkout\..*Payment" # classes ending in Payment spektrum -s spec -p "re:checkout\..*Payment.*" # and those only containing it
-t selects by case name, as a comma-delimited list. A name matches in every spec that
defines it, so one name can run in several places:
spektrum -s spec -t adds_item_to_cart spektrum -s spec -t adds_item_to_cart,removes_item_from_cart
-t accepts re: as well. That pattern is anchored at both ends, and is tried
against the method name and against the spaced form Spektrum prints in its output:
spektrum -s spec -t "re:adds item to cart"
-m selects by the @metadata decorator and --exclude-by-metadata is its
complement, which is how you mark a slow or environment-dependent group and skip it while
working locally:
spektrum -s spec -m suite=smoke spektrum -s spec --exclude-by-metadata suite=slow
--dry-run reports the tree without executing anything or evaluating a single assertion
-- the quickest way to confirm a filter selects what you meant before you wait on a run.
Spektrum writes xunit XML for CI systems that consume it:
spektrum -s path/to/specs --xunit-results results.xml
--coverage records coverage data for the code under test into .coverage, for a
following coverage report; it prints nothing itself. --live serves a live progress
view over HTTP while the run is in flight. Spektrum ships console, xunit, live and TestRail
reporters, the last through the --tr-* options.
See the contributing guide for setup, the test suites, and what a change needs to prove before review.
Spektrum is published to PyPI at https://pypi.org/project/Spektrum/:
pip install spektrum
Releases are cut every week or two, and only when something has merged — a quiet fortnight produces no release.
Release notes live in
the documentation. The section at the top, Next Release, is the accumulating draft of
the next version: if your PR changes behaviour, add your entry there as part of it. The
release renames that section to the version it becomes, so the note written at review time
is the note that ships.
Maintainers: the procedure is in docs/maintenance/index.rst, and
./tools/release.sh runs it.