#35414: Issue with AsyncClient ignoring default headers compared to synchronous
Client
-------------------------------------+-------------------------------------
Reporter: 설원준(Wonjoon | Owner: nobody
Seol)/Dispatch squad |
Type: Bug | Status: new
Component: HTTP handling | Version: 5.0
Severity: Normal | Resolution:
Keywords: AsyncClient, | Triage Stage:
ASGIRequest | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Description changed by 설원준(Wonjoon Seol)/Dispatch squad:
Old description:
> Currently, there is an inconsistency between Django's asynchronous
> AsyncClient and its synchronous counterpart Client regarding the handling
> of default headers. While the synchronous Client correctly includes
> default headers, the asynchronous AsyncClient ignores them. This behavior
> leads to discrepancies when utilizing fixtures with default headers,
> causing tests to fail unexpectedly.
>
> == Reproduction Steps:
>
> - Set up a fixture with default headers for both synchronous and
> asynchronous clients.
> - Utilize the fixtures in test cases and observe the behavior.
> - Notice that the synchronous client includes default headers as
> expected, while the asynchronous client does not.
>
> == Code Snippets:
> {{{
> @pytest.fixture(scope="session")
> def jwt_token(token_payload: dict[str, Any]) -> str:
> return jwt.encode({"abc", '"abc"}, key='123', algorithm="HS256")
>
> # this passes HTTP_AUTHORIZATION default header
> @pytest.fixture(scope="session")
> def sync_client_with_token(jwt_token) -> Generator[Client, None, None]:
> yield Client(HTTP_AUTHORIZATION=f"Bearer {jwt_token}")
>
> # this does not
> @pytest.fixture(scope="session")
> async def async_client_with_token(jwt_token) ->
> AsyncIterator[AsyncClient]:
> async_client = AsyncClient(HTTP_AUTHORIZATION=f"Bearer {jwt_token}")
> # async_client.defaults["AUTHORIZATION"] = f"Bearer {jwt_token}"
> yield async_client
> }}}
>
> AsyncRequestFactory.generic() does not currently check if self.defaults
> exists and ASGIRequest only check hard-coded header names in __init__()
> method, effectively ignoring rest of the self.scope values.
>
> Note that while RequestFactory.generic() method does not check whether
> self.defaults exist but WSGIRequest receives default values via
> ._base_environ() method when creating WSGIRequest instance.
>
> == Proposed Solutions:
>
> **Fix Method 1: Modify AsyncRequestFactory.generic() method**
> {{{
> class AsyncRequestFactory(RequestFactory):
> def generic(
> self,
> method,
> path,
> data="",
> content_type="application/octet-stream",
> secure=False,
> *,
> headers=None,
> **extra,
> ):
> """Construct an arbitrary HTTP request."""
> parsed = urlparse(str(path)) # path can be lazy.
> data = force_bytes(data, settings.DEFAULT_CHARSET)
> s = {
> "method": method,
> "path": self._get_path(parsed),
> "server": ("127.0.0.1", "443" if secure else "80"),
> "scheme": "https" if secure else "http",
> "headers": [(b"host", b"testserver")],
> }
> if self.defaults: # <- added
> extra = {**self.defaults, **extra} <- added
> if data:
> s["headers"].extend(
> [
> (b"content-length", str(len(data)).encode("ascii")),
> (b"content-type", content_type.encode("ascii")),
> ]
> )
> s["_body_file"] = FakePayload(data)
> ...
> }}}
>
> **Fix Method 2: Modify ASGIRequest**
> Alternatively, the ASGIRequest class can be adjusted to include default
> headers in the META attribute during initialisation.
>
> {{{
> class ASGIRequest(HttpRequest):
>
> def __init__(self, scope, body_file):
> self.scope = scope
> ...
> if isinstance(query_string, bytes):
> query_string = query_string.decode()
> self.META = {
> **self.scope, <- # Added
> "REQUEST_METHOD": self.method,
> "QUERY_STRING": query_string,
> "SCRIPT_NAME": self.script_name,
> "PATH_INFO": self.path_info,
> }}}
>
> This would be simliar to WSGIRequest, where self.META = environ is set
> during the init phase.
> However, it's worth noting that WSGI has a different META format.
>
> **asgi META **
> {{{
> {'asgi': {'version': '3.0'}, 'type': 'http', 'http_version': '1.1',
> 'client': ['127.0.0.1', 0], 'server': ('127.0.0.1', '80'), 'scheme':
> 'http', 'method': 'GET', 'headers': [(b'host', b'testserver'),
> (b'cookie', b'')], 'HTTP_AUTHORIZATION': 'Bearer
> eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE3MTQ0NDg2NzcsImV4cCI6MTcxNDQ1MjI3Nywic3ViX2lkIjoiMSIsInBsYXRmb3JtIjoiVk1TLXN0YWdpbmciLCJiYXNlX3VybCI6Imh0dHBzOi8vc3RhZ2luZy1jZW8tcG9ydGFsLWFwaS55b2dpeW8uY28ua3IvIiwicm9sZSI6InN0YWZmIiwidXNlcl9pZCI6IjEiLCJzdGFmZl9ncm91cF9pZCI6bnVsbH0
> .WWubd4iOUnsqbWO0ba-8mAsCCk3QDbBONvB_nznZQsk', 'path':
> '/campaign/v1/campaigns/1/items/', 'query_string':
> 'date_type=created_at&date_from=2024-04-28+03%3A44%3A37.484747%2B00%3A00&date_to=2024-05-02+03%3A44%3A37.484759%2B00%3A00',
> 'REQUEST_METHOD': 'GET', 'QUERY_STRING':
> 'date_type=created_at&date_from=2024-04-28+03%3A44%3A37.484747%2B00%3A00&date_to=2024-05-02+03%3A44%3A37.484759%2B00%3A00',
> 'SCRIPT_NAME': '', 'PATH_INFO': '/campaign/v1/campaigns/1/items/',
> 'wsgi.multithread': True, 'wsgi.multiprocess': True, 'REMOTE_ADDR':
> '127.0.0.1', 'REMOTE_HOST': '127.0.0.1', 'REMOTE_PORT': 0, 'SERVER_NAME':
> '127.0.0.1', 'SERVER_PORT': '80', 'HTTP_HOST': 'testserver',
> 'HTTP_COOKIE': ''}
> }}}
>
> ASGI META has separate 'headers' but the custom headers are not added
> there.
>
> **wsgi META**
> {{{
> {'HTTP_COOKIE': '', 'PATH_INFO': '/campaign/v1/campaigns/1/items/',
> 'REMOTE_ADDR': '127.0.0.1', 'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '',
> 'SERVER_NAME': 'testserver', 'SERVER_PORT': '80', 'SERVER_PROTOCOL':
> 'HTTP/1.1', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http',
> 'wsgi.input': <django.test.client.FakePayload object at 0x1061b0d90>,
> 'wsgi.errors': <_io.BytesIO object at 0x104f42020>, 'wsgi.multiprocess':
> True, 'wsgi.multithread': False, 'wsgi.run_once': False,
> 'HTTP_AUTHORIZATION': 'Bearer
> eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE3MTQ0NDg3OTQsImV4cCI6MTcxNDQ1MjM5NCwic3ViX2lkIjoiMSIsInBsYXRmb3JtIjoiVk1TLXN0YWdpbmciLCJiYXNlX3VybCI6Imh0dHBzOi8vc3RhZ2luZy1jZW8tcG9ydGFsLWFwaS55b2dpeW8uY28ua3IvIiwicm9sZSI6InN0YWZmIiwidXNlcl9pZCI6IjEiLCJzdGFmZl9ncm91cF9pZCI6bnVsbH0.MkbgS1zDaMLdDMLC0_Jpe_2O7VBtJD8km70Y0KlUb4g',
> 'QUERY_STRING':
> 'date_type=created_at&date_from=2024-04-28+03%3A46%3A35.172175%2B00%3A00&date_to=2024-05-02+03%3A46%3A35.172190%2B00%3A00'}
> }}}
>
> Addressing this inconsistency will ensure that the behaviour of both
> synchronous and asynchronous clients remains consistent and predictable
> across Django applications.
>
> Thanks.
New description:
Currently, there is an inconsistency between Django's asynchronous
AsyncClient and its synchronous counterpart Client regarding the handling
of default headers. While the synchronous Client correctly includes
default headers, the asynchronous AsyncClient ignores them. This behavior
leads to discrepancies when utilizing fixtures with default headers,
causing tests to fail unexpectedly.
== Reproduction Steps:
- Set up a fixture with default headers for both synchronous and
asynchronous clients.
- Utilize the fixtures in test cases and observe the behavior.
- Notice that the synchronous client includes default headers as expected,
while the asynchronous client does not.
== Code Snippets:
{{{
@pytest.fixture(scope="session")
def jwt_token(token_payload: dict[str, Any]) -> str:
return jwt.encode({"abc", '"abc"}, key='123', algorithm="HS256")
# this passes HTTP_AUTHORIZATION default header
@pytest.fixture(scope="session")
def sync_client_with_token(jwt_token) -> Generator[Client, None, None]:
yield Client(HTTP_AUTHORIZATION=f"Bearer {jwt_token}")
# this does not
@pytest.fixture(scope="session")
async def async_client_with_token(jwt_token) ->
AsyncIterator[AsyncClient]:
async_client = AsyncClient(HTTP_AUTHORIZATION=f"Bearer {jwt_token}")
# async_client.defaults["AUTHORIZATION"] = f"Bearer {jwt_token}"
yield async_client
}}}
AsyncRequestFactory.generic() does not currently check if self.defaults
exists and ASGIRequest only check hard-coded header names in __init__()
method, effectively ignoring rest of the self.scope values.
Note that while RequestFactory.generic() method does not check whether
self.defaults exist but WSGIRequest receives default values via
._base_environ() method when creating WSGIRequest instance.
== Proposed Solutions:
**Fix Method 1: Modify AsyncRequestFactory.generic() method**
{{{
class AsyncRequestFactory(RequestFactory):
def generic(
self,
method,
path,
data="",
content_type="application/octet-stream",
secure=False,
*,
headers=None,
**extra,
):
"""Construct an arbitrary HTTP request."""
parsed = urlparse(str(path)) # path can be lazy.
data = force_bytes(data, settings.DEFAULT_CHARSET)
s = {
"method": method,
"path": self._get_path(parsed),
"server": ("127.0.0.1", "443" if secure else "80"),
"scheme": "https" if secure else "http",
"headers": [(b"host", b"testserver")],
}
if self.defaults: # <- added
extra = {**self.defaults, **extra} <- added
if data:
s["headers"].extend(
[
(b"content-length", str(len(data)).encode("ascii")),
(b"content-type", content_type.encode("ascii")),
]
)
s["_body_file"] = FakePayload(data)
...
}}}
**Fix Method 2: Modify ASGIRequest**
Alternatively, the ASGIRequest class can be adjusted to include default
headers in the META attribute during initialisation.
{{{
class ASGIRequest(HttpRequest):
def __init__(self, scope, body_file):
self.scope = scope
...
if isinstance(query_string, bytes):
query_string = query_string.decode()
self.META = {
**self.scope, <- # Added
"REQUEST_METHOD": self.method,
"QUERY_STRING": query_string,
"SCRIPT_NAME": self.script_name,
"PATH_INFO": self.path_info,
}}}
This would be simliar to WSGIRequest, where self.META = environ is set
during the init phase.
However, it's worth noting that WSGI has a different META format.
**asgi META **
{{{
{'asgi': {'version': '3.0'}, 'type': 'http', 'http_version': '1.1',
'client': ['127.0.0.1', 0], 'server': ('127.0.0.1', '80'), 'scheme':
'http', 'method': 'GET', 'headers': [(b'host', b'testserver'), (b'cookie',
b'')], 'HTTP_AUTHORIZATION': 'Bearer ...', 'path': '/campaigns/',
'query_string':
'date_type=created_at&date_from=2024-04-28+03%3A44%3A37.484747%2B00%3A00&date_to=2024-05-02+03%3A44%3A37.484759%2B00%3A00',
'REQUEST_METHOD': 'GET', 'QUERY_STRING':
'date_type=created_at&date_from=2024-04-28+03%3A44%3A37.484747%2B00%3A00&date_to=2024-05-02+03%3A44%3A37.484759%2B00%3A00',
'SCRIPT_NAME': '', 'PATH_INFO': '/campaign/v1/campaigns/1/items/',
'wsgi.multithread': True, 'wsgi.multiprocess': True, 'REMOTE_ADDR':
'127.0.0.1', 'REMOTE_HOST': '127.0.0.1', 'REMOTE_PORT': 0, 'SERVER_NAME':
'127.0.0.1', 'SERVER_PORT': '80', 'HTTP_HOST': 'testserver',
'HTTP_COOKIE': ''}
}}}
ASGI META has separate 'headers' but the custom headers are not added
there.
**wsgi META**
{{{
{'HTTP_COOKIE': '', 'PATH_INFO': '/campaigns/', 'REMOTE_ADDR':
'127.0.0.1', 'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'SERVER_NAME':
'testserver', 'SERVER_PORT': '80', 'SERVER_PROTOCOL': 'HTTP/1.1',
'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.input':
<django.test.client.FakePayload object at 0x1061b0d90>, 'wsgi.errors':
<_io.BytesIO object at 0x104f42020>, 'wsgi.multiprocess': True,
'wsgi.multithread': False, 'wsgi.run_once': False, 'HTTP_AUTHORIZATION':
'Bearer ....', 'QUERY_STRING':
'date_type=created_at&date_from=2024-04-28+03%3A46%3A35.172175%2B00%3A00&date_to=2024-05-02+03%3A46%3A35.172190%2B00%3A00'}
}}}
Addressing this inconsistency will ensure that the behaviour of both
synchronous and asynchronous clients remains consistent and predictable
across Django applications.
Thanks.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/35414#comment:3>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/0107018f2d391ce1-b38ba212-149e-473d-87f6-7b15297431c3-000000%40eu-central-1.amazonses.com.
