Getting Started with HTTPX, Part 2: pytest and pytest_httpx

In Part 1, we built a simple Wikipedia search tool using Python and HTTPX.

Now, let’s use pytest and write some tests.

Ill-advised: test against remote endpoints

We could test directly against Wikipedia’s live servers. The contents of tests/test_pypedia.py could be something like this:

import pypedia.synchronous


def test_sync_search():
    response = pypedia.synchronous.search("incategory:Python_(programming_language)")
    assert response.status_code == 200
    assert "Guido" in response.text

Try it out:

poetry run pytest

It works! Probably.

However, this strategy could cause pain for any of the following reasons:

• The information could change (a wiki editor could fiendishly change Guido’s name, for instance) and your tests would fail for reasons that have nothing to do with your code.
• Wikipedia could go down temporarily and your tests would fail for reasons that have nothing to do with your code.
• Wikipedia could institute API limits and you run too many tests per minute, or decide to charge money and your funds run dry, and your tests would fail for reasons that have nothing to do with your code.

Sensing a theme?

Rather than test against a live server, we need to be able to test against a “mock” endpoint that returns predictable data. That way, what we test is the code we write, rather than the capabilities of someone else’s server.

Installing pytest_httpx

It is certainly possible to build your own HTTP mock, but I am grateful for packages that make it easy.

RESPX is one of these.

Another option is pytest_httpx which is built for pytest and works well now. This is the package I have chosen.

Before proceeding, we need to both upgrade pytest and install pytest_httpx:

poetry add -D pytest@latest pytest_httpx

Note that the -D flag is used to specify these are development dependency, not packages the code needs in production.

Mock the request

To make a mock request, whatever we are testing against needs to be specified in the mock. For this, the httpx_mock.add_response() function is used.

By default, it will mock a 200 HTTP response with an empty body.

If testing to make sure the URL is formed correctly, pass the url to httpx_mock.add_response() with the url parameter.

If testing the content returned by the request, pass the content to httpx_mock.add_response(). The data parameter accepts arbitrary text, while the json parameter will first convert the passed Python object to JSON, as seen below.

import pypedia.synchronous

ARTICLES = {
    "pages": [
        {
            "id": 23862,
            "key": "Python_(programming_language)",
            "title": "Python (programming language)",
            "excerpt": "Python is an interpreted, high-level, general-purpose programming language.",
            "description": "An interpreted, high-level, general-purpose programming language.",
        },
        {
            "id": 226402,
            "key": "Guido_van_Rossum",
            "title": "Guido van Rossum",
            "excerpt": "Guido van Rossum; the creator of the Python programming language",
            "description": "Dutch programmer and creator of Python",
        },
    ]
}
SEARCH = "incategory:Python_(programming_language)"
URL = "https://en.wikipedia.org/w/rest.php/v1/search/page?q=incategory%3APython_%28programming_language%29&limit=100"


def test_mock_sync_search(httpx_mock):
    httpx_mock.add_response(url=URL, json=ARTICLES)
    response = pypedia.synchronous.search(SEARCH)
    assert response.status_code == 200
    assert "Guido" in response.text

The ARTICLES dict represents a Wikipedia response. However, I removed some info that I didn’t need to test. In fact, I could go much further and remove everything I don’t need now. Instead, I included such keys as id, key, and description in case I want to use those in the future, even though there is nothing in the code that utilizes that particular data now. In other words, a little laziness now allows for a little laziness later.

Run pytest

Does the test pass?

poetry run pytest

We have a successful, synchronous Wikipedia client.

In the next article, we take that synchronous client we wrote in the previous article, and tweak it to run asynchronously. And, yes, pytest_httpx can also be used to test asynchronous functionality.