Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Truststore #3409

Open
wants to merge 21 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/test-suite.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:

strategy:
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
python-version: ["3.10", "3.11", "3.12", "3.13"]

steps:
- uses: "actions/checkout@v4"
Expand Down
8 changes: 5 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,17 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## 0.28.0 (...)

TODO... writeup `truststore` switch & 3.10+ requirement.

The 0.28 release includes a limited set of backwards incompatible changes.

**Backwards incompatible changes**:

SSL configuration has been significantly simplified.

* The `verify` argument no longer accepts string arguments.
* The `cert` argument has now been removed.
* The `SSL_CERT_FILE` and `SSL_CERT_DIR` environment variables are no longer automatically used.
* The `verify` argument no longer accepts string arguments. Explicitly specified certificate stores can still be enabled through the SSL configuration API.
* The `cert` argument has now been removed. Client side certificates can still be enabled through the SSL configuration API.
* The `SSL_CERT_FILE` and `SSL_CERT_DIR` environment variables are no longer used. These environment variables can be enabled manually although should be obsoleted by our switch to `truststore`.

For users of the standard `verify=True` or `verify=False` cases this should require no changes.

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ Or, to include the optional HTTP/2 support, use:
$ pip install httpx[http2]
```

HTTPX requires Python 3.8+.
HTTPX requires Python 3.10+.

## Documentation

Expand All @@ -125,7 +125,7 @@ The HTTPX project relies on these excellent libraries:

* `httpcore` - The underlying transport implementation for `httpx`.
* `h11` - HTTP/1.1 support.
* `certifi` - SSL certificates.
* `truststore` - System SSL certificates.
* `idna` - Internationalized domain name support.
* `sniffio` - Async library autodetection.

Expand Down
50 changes: 15 additions & 35 deletions docs/advanced/ssl.md
Original file line number Diff line number Diff line change
@@ -1,62 +1,42 @@
When making a request over HTTPS, HTTPX needs to verify the identity of the requested host. To do this, it uses a bundle of SSL certificates (a.k.a. CA bundle) delivered by a trusted certificate authority (CA).
When making a request over HTTPS we need to verify the identity of the requested host. We rely on the [`truststore`](https://truststore.readthedocs.io/en/latest/) package to load the system certificates, ensuring that `httpx` has the same behaviour on SSL sites as your browser.

### Enabling and disabling verification
### SSL verification

By default httpx will verify HTTPS connections, and raise an error for invalid SSL cases...

```pycon
```python
>>> httpx.get("https://expired.badssl.com/")
httpx.ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)
```

You can disable SSL verification completely and allow insecure requests...
If you're confident that you want to visit a site with an invalid certificate you can disable SSL verification completely...

```pycon
```python
>>> httpx.get("https://expired.badssl.com/", verify=False)
<Response [200 OK]>
```

### Configuring client instances
### Custom SSL configurations

If you're using a `Client()` instance you should pass any `verify=<...>` configuration when instantiating the client.
If you're using a `Client()` instance you can pass the `verify=<...>` configuration when instantiating the client.

By default the [certifi CA bundle](https://certifiio.readthedocs.io/en/latest/) is used for SSL verification.
```python
>>> client = httpx.Client(verify=True)
```

For more complex configurations you can pass an [SSL Context](https://docs.python.org/3/library/ssl.html) instance...

```python
import certifi
import httpx
import ssl
tomchristie marked this conversation as resolved.
Show resolved Hide resolved
import certifi

# This SSL context is equivelent to the default `verify=True`.
# Use certifi for certificate validation, rather than the system truststore.
ctx = ssl.create_default_context(cafile=certifi.where())
client = httpx.Client(verify=ctx)
```

Using [the `truststore` package](https://truststore.readthedocs.io/) to support system certificate stores...

```python
import ssl
import truststore
import httpx

# Use system certificate stores.
ctx = truststore.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
client = httpx.Client(verify=ctx)
```

Loding an alternative certificate verification store using [the standard SSL context API](https://docs.python.org/3/library/ssl.html)...

```python
import httpx
import ssl

# Use an explicitly configured certificate store.
ctx = ssl.create_default_context(cafile="path/to/certs.pem") # Either cafile or capath.
client = httpx.Client(verify=ctx)
```

### Client side certificates

Client side certificates allow a remote server to verify the client. They tend to be used within private organizations to authenticate requests to remote servers.
Expand All @@ -71,9 +51,9 @@ client = httpx.Client(verify=ctx)

### Working with `SSL_CERT_FILE` and `SSL_CERT_DIR`

Unlike `requests`, the `httpx` package does not automatically pull in [the environment variables `SSL_CERT_FILE` or `SSL_CERT_DIR`](https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_default_verify_paths.html). If you want to use these they need to be enabled explicitly.
Unlike `requests`, the `httpx` package does not automatically pull in [the environment variables `SSL_CERT_FILE` or `SSL_CERT_DIR`](https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_default_verify_paths.html).

For example...
These environment variables shouldn't be necessary since they're obsoleted by `truststore`. They can be enabled if required like so...

```python
# Use `SSL_CERT_FILE` or `SSL_CERT_DIR` if configured.
Expand All @@ -87,7 +67,7 @@ client = httpx.Client(verify=ctx)

### Making HTTPS requests to a local server

When making requests to local servers, such as a development server running on `localhost`, you will typically be using unencrypted HTTP connections.
When making requests to local servers such as a development server running on `localhost`, you will typically be using unencrypted HTTP connections.

If you do need to make HTTPS connections to a local server, for example to test an HTTPS-only service, you will need to create and use your own certificates. Here's one way to do it...

Expand Down
4 changes: 2 additions & 2 deletions httpx/_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@ class UnsetType:
def create_ssl_context(verify: ssl.SSLContext | bool = True) -> ssl.SSLContext:
import ssl

import certifi
import truststore

if verify is True:
return ssl.create_default_context(cafile=certifi.where())
return truststore.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
elif verify is False:
ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
ssl_context.check_hostname = False
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ classifiers = [
"Topic :: Internet :: WWW/HTTP",
]
dependencies = [
"certifi",
"truststore==0.10.0",
"httpcore==1.*",
"anyio",
"idna",
Expand Down
23 changes: 0 additions & 23 deletions tests/test_config.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,5 @@
import ssl
import typing
from pathlib import Path

import certifi
import pytest

import httpx
Expand All @@ -20,26 +17,6 @@ def test_load_ssl_config_verify_non_existing_file():
context.load_verify_locations(cafile="/path/to/nowhere")


def test_load_ssl_with_keylog(monkeypatch: typing.Any) -> None:
monkeypatch.setenv("SSLKEYLOGFILE", "test")
context = httpx.create_ssl_context()
assert context.keylog_filename == "test"


def test_load_ssl_config_verify_existing_file():
context = httpx.create_ssl_context()
context.load_verify_locations(capath=certifi.where())
assert context.verify_mode == ssl.VerifyMode.CERT_REQUIRED
assert context.check_hostname is True


def test_load_ssl_config_verify_directory():
context = httpx.create_ssl_context()
context.load_verify_locations(capath=Path(certifi.where()).parent)
assert context.verify_mode == ssl.VerifyMode.CERT_REQUIRED
assert context.check_hostname is True


def test_load_ssl_config_cert_and_key(cert_pem_file, cert_private_key_file):
context = httpx.create_ssl_context()
context.load_cert_chain(cert_pem_file, cert_private_key_file)
Expand Down