pytest_container

Testing Container Images with Python and Pytest

Dan Čermák

  oSC25

CC BY 4.0

who -u

Dan Čermák

Software Developer @SUSE
i3 SIG, Package maintainer
Developer Tools, Testing and Documentation, Home Automation
https://dancermak.name
dcermak
@Defolos@mastodon.social

Why not /bin/sh?

When shell scripts are:

  • portable / run everywhere
  • fast 🏎️
  • everyone understands them

Until they aren't 💥

Why pytest_container?

  • shell scripts are brittle
  • automatically pull, build, launch & cleanup containers
  • use testinfra for convenience
  • parallel test execution via pytest-xdist
  • find & bind free ports
  • create & destroy pods
  • create and cleanup container volumes & bind mounts
  • run the same test on multiple container images
  • supports docker and podman transparently
  • works with Python 3.6+ and on x86_64, aarch64, s390x, ppcle64

Basic Example

TW = Container(
    url="registry.opensuse.org/opensuse/tumbleweed:latest"
)

@pytest.mark.parametrize("container", [TW], indirect=True)
def test_etc_os_release_present(container: ContainerData):
    assert container.connection.file(
        "/etc/os-release"
    ).exists

Use Cases

  • system tests of base containers
  • test applications inside containers
  • run tests of your application on multiple OS'

pytest fixtures

  • fixtures for setup & teardown 
def test_ehlo(smtp_connection):
    response, msg = smtp_connection.ehlo()
    assert response == 250

@pytest.mark.parametrize("x", [0, 1])
@pytest.mark.parametrize("y", [2, 3])
def test_foo(x: int, y: int):
    # do something with x & y here

Usage examples

Build a new container for tests

WEB_SERVER = DerivedContainer(
    base="registry.opensuse.org/opensuse/leap:latest",
    containerfile="""RUN zypper -n in python3 \
    && echo "Hello Green World!" > index.html
ENTRYPOINT ["/usr/bin/python3", "-m", "http.server"]
"""
)

@pytest.mark.parametrize(
    "container", [WEB_SERVER], indirect=True
)
def test_python_installed(container: ContainerData):
    assert container.connection.package(
        "python3"
    ).is_installed

Dependencies between containers

TW = Container(
    url="registry.opensuse.org/opensuse/tumbleweed:latest"
)
NGINX = DerivedContainer(
    base=TW,
    containerfile="RUN zypper -n in nginx",
)
NGINX_DEBUG = DerivedContainer(
    base=NGINX,
    containerfile="RUN zypper -n in gdb nginx-debuginfo"
)

CONTAINER_IMAGES=[NGINX_DEBUG]

def test_nginx(auto_container): ...

Get a free port on the host

WEB_SERVER = DerivedContainer(
    # snip
    forwarded_ports=[PortForwarding(container_port=8000)],
)

@pytest.mark.parametrize(
    "container", [WEB_SERVER], indirect=True
)
def test_port_forward(container: ContainerData, host):
    cmd = (
        "curl --fail localhost:"
        + str(container.forwarded_ports[0].host_port)
    )
    host.check_output(cmd)

HEALTHCHECK

WEB_SERVER = DerivedContainer(
    # snip
    containerfile="""
ENTRYPOINT ["/usr/bin/python3", "-m", "http.server"]
HEALTHCHECK CMD curl --fail http://0.0.0.0:8000""",
)


@pytest.mark.parametrize("container", [WEB_SERVER], indirect=True)
def test_server_up(container, container_runtime):
    assert (
        container_runtime.get_container_health(
            container.container_id
        ) == ContainerHealth.HEALTHY
    )

Create a Pod

MEDIAWIKI_FPM_POD = Pod(
    containers=[MEDIAWIKI_FPM_CONTAINER, NGINX_FPM_PROXY],
    forwarded_ports=[PortForwarding(container_port=80)],
)

@pytest.mark.parametrize(
    "pod", [MEDIAWIKI_FPM_POD], indirect=True
)
def test_port_forward(pod: PodData, host):
    cmd = (
        "curl --fail localhost:"
        + str(pod.forwarded_ports[0].host_port)
    )
    host.check_output(cmd)

Run mutable tests

use the container_per_test fixture: 

@pytest.mark.parametrize(
    "container_per_test", [TW], indirect=True
)
def test_rm_rf(container_per_test):
    container_per_test.connection.check_output("rm -rf /")


@pytest.mark.parametrize(
    "container_per_test", [TW], indirect=True
)
def test_uninstall_zypper(container_per_test):
    container_per_test.connection.check_output(
        "rpm -e --nodeps zypper"
    )

🔎 Inspect containers

@pytest.mark.parametrize(
    "container", [MY_IMAGE], indirect=True
)
def test_inspect(container: ContainerData):
    inspect = container.inspect

    assert inspect.config.user == "me"
    assert inspect.config.cmd == ["/bin/sh"]

    assert (
        "HOME" in inspect.config.env
        and inspect.config.env["HOME"] == "/src/"
    )

Container Volumes

Bind mounts 

ROOTDIR_BIND_MOUNTED = DerivedContainer(
    base="registry.opensuse.org/opensuse/tumbleweed",
    volume_mounts=[
        BindMount("/src/", host_path=get_rootdir())
    ],
)


@pytest.mark.parametrize(
    "container", [ROOTDIR_BIND_MOUNTED], indirect=True
)
def test_bind_mount_cwd(container: ContainerData):
    vol = container.container.volume_mounts[0]
    assert container.connection.file("/src/").exists

Container volumes 

WITH_VAR_LOG_VOLUME = DerivedContainer(
    base="registry.opensuse.org/opensuse/tumbleweed",
    volume_mounts=[ContainerVolume("/var/log/")],
)

Use the same container globally

use the auto_container / auto_container_per_test fixtures: 

CONTAINER_IMAGES = [TW, LEAP, SLE]


def test_etc_os_release(auto_container): ...


def test_zypper_rm_works(auto_container_per_test): ...

Pick the Container Engine

export CONTAINER_RUNTIME=docker
pytest -vv

Run tests in parallel

pip install pytest-xdist
# or
poetry add --group dev pytest-xdist

pytest -vv -- -n auto

🧹 Automatic cleanup

  • containers
  • volumes
  • pods
  • temporary directories
  • ⚠️Images and intermediate layers are retained ⚠️

Roadmap

  • better entrypoint testing (#206)
  • support for foreign architecture containers (#183)
  • improved multistage build support (#149)

Users

Thanks!

Give it a try!

dcermak/pytest_container

dcermak.github.io/pytest_container

dcermak.github.io/pytest_container-presentation

Questions?

Answers!

Suggestions?

👉 github.com/dcermak/pytest_container/issues