pytest-flake8-path

A pytest fixture for testing flake8 plugins.

A quick example:

def test_simple_run(flake8_path):
    (flake8_path / "example.py").write_text("x  = 1\n")

    result = flake8_path.run_flake8()

    assert result.out_lines == [
        "./example.py:1:2: E221 multiple spaces before operator"
    ]

Working on a Django project? Check out my book Speed Up Your Django Tests which covers loads of ways to write faster, more accurate tests.

Installation

Use pip:

python -m pip install pytest-flake8-path

Python 3.9 to 3.13 supported.

API

flake8_path fixture

A pytest fixture that wraps Pytest's built-in tmp_path fixture (docs), to create a temporary directory, allow adding files, and running flake8. It acts like a pathlib.Path object, with one extra method.

If you're using this to test a flake8 plugin, make sure flake8 is picking up your plugin during tests. Normally this is done with a setup.cfg entrypoint, which makes tox the easiest way to guarantee this is ready as it will install your project before running tests.

flake8dir.run_flake8(extra_args: list[str] | None = None) -> Flake8Result

Runs flake8 and returns a Flake8Result representing the results.

extra_args may be a list of extra flags to pass to flake8, for example passing ["--ignore", "E101"]. Note some arguments are already passed to ensure Flake8 runs in an isolated manner - see source.

Flake8Result

Represents the parsed output of a flake8 run.

Flake8Result.out: str

The full string of output (stdout) generated by flake8.

Flake8Result.err: str

The full string of error output (stderr) generated by flake8.

Flake8Result.exit_code: int

The exit code that the flake8 run exited with.

Flake8Result.out_lines: list[str]

A list of individual lines of output, without trailing newlines. This is the most useful tool for making assertions against.

On Windows, file paths are normalized into the Unix format (\ is replaced with /). This allows test suites to run the same on all operating systems.

For example, given a result you can check for a particular line being output:

result = flake8_path.run_flake8()
expected = "./example.py:1:2: E221 multiple spaces before operator"
assert expected in result.out_lines

Flake8Result.err_lines: list[str]

Like out_lines, but for error output.

Examples

Basic

Using Path.write_text() (docs) we can create an example.py file with code. Then with flake8_path.run_flake8() we can run flake8 and check for an expected error:

def test_simple(flake8_path):
    (flake8_path / "example.py").write_text("x  = 1\n")

    result = flake8_path.run_flake8()

    assert result.out_lines == [
        "./example.py:1:2: E221 multiple spaces before operator"
    ]
    assert result.err_lines == []
    assert result.exit_code == 1

With dedent

The standard library’s textwrap.dedent() (docs) is useful for including multi-line files. Use a triple quoted multi-line string, with an initial backslash to prevent a blank first line:

def test_multi_line(flake8_path):
    (flake8_path / "example.py").write_text(
        dedent(
            """\
            x  = 1
            y  = 2
            """
        )
    )

    result = flake8_path.run_flake8()

    assert result.out_lines == [
        "./example.py:1:2: E221 multiple spaces before operator",
        "./example.py:2:2: E221 multiple spaces before operator",
    ]
    assert result.err_lines == []
    assert result.exit_code == 1

Configuring flake8

Write a setup.cfg file to configure flake8 before running it:

def test_with_setup_cfg(flake8_path):
    (flake8_path / "setup.cfg").write_text(
        dedent(
            """\
            [flake8]
            ignore = E221
            """
        )
    )
    (flake8_path / "example.py").write_text("x  = 1\n")

    result = flake8_path.run_flake8()

    assert result.out_lines == []
    assert result.err_lines == []
    assert result.exit_code == 0

History

pytest-flake8-path is the successor to pytest-flake8dir. pytest-flake8dir was based upon pytest’s tmpdir fixture, which returned a legacy py.path.local object. Since version 3.9.0, pytest has provided the tmp_path fixture, which returns a standard library pathlib.Path object. pytest-flake8-path is a rewrite of pytest-flake8dir to use tmp_path instead of tmpdir.