#35414: Issue with AsyncClient ignoring default headers compared to synchronous Client -------------------------------------+------------------------------------- Reporter: wonjoonSeol-WS | Owner: nobody Type: Bug | Status: new Component: HTTP handling | Version: 5.0 Severity: Normal | Resolution: Keywords: AsyncClient, | Triage Stage: ASGIRequest | Unreviewed Has patch: 0 | Needs documentation: 0 Needs tests: 0 | Patch needs improvement: 0 Easy pickings: 0 | UI/UX: 0 -------------------------------------+------------------------------------- Description changed by wonjoonSeol-WS:
Old description: > == Description: > Currently, there is an inconsistency between Django's asynchronous > AsyncClient and its synchronous counterpart Client regarding the handling > of default headers. While the synchronous Client correctly includes > default headers, the asynchronous AsyncClient ignores them. This behavior > leads to discrepancies when utilizing fixtures with default headers, > causing tests to fail unexpectedly. > > == Reproduction Steps: > > Set up a fixture with default headers for both synchronous and > asynchronous clients. > Utilize the fixtures in test cases and observe the behavior. > Notice that the synchronous client includes default headers as expected, > while the asynchronous client does not. > > == Code Snippets: > {{{ > @pytest.fixture(scope="session") > def jwt_token(token_payload: dict[str, Any]) -> str: > return jwt.encode({"abc", '"abc"}, key='123', algorithm="HS256") > > # this passes HTTP_AUTHORIZATION default header > @pytest.fixture(scope="session") > def sync_client_with_token(jwt_token) -> Generator[Client, None, None]: > yield Client(HTTP_AUTHORIZATION=f"Bearer {jwt_token}") > > # this does not > @pytest.fixture(scope="session") > async def async_client_with_token(jwt_token) -> > AsyncIterator[AsyncClient]: > async_client = AsyncClient(HTTP_AUTHORIZATION=f"Bearer {jwt_token}") > # async_client.defaults["AUTHORIZATION"] = f"Bearer {jwt_token}" > yield async_client > }}} > > AsyncRequestFactory.generic() does not currently check if self.defaults > exists and ASGIRequest only check hard-coded header names in __init__() > method, effectively ignoring rest of the self.scope values. > > Note that while RequestFactory.generic() method does not check whether > self.defaults exist but WSGIRequest receives default values via > ._base_environ() method when creating WSGIRequest instance. > > == Proposed Solutions: > > **Fix Method 1: Modify AsyncRequestFactory.generic() method** > {{{ > class AsyncRequestFactory(RequestFactory): > def generic( > self, > method, > path, > data="", > content_type="application/octet-stream", > secure=False, > *, > headers=None, > **extra, > ): > """Construct an arbitrary HTTP request.""" > parsed = urlparse(str(path)) # path can be lazy. > data = force_bytes(data, settings.DEFAULT_CHARSET) > s = { > "method": method, > "path": self._get_path(parsed), > "server": ("127.0.0.1", "443" if secure else "80"), > "scheme": "https" if secure else "http", > "headers": [(b"host", b"testserver")], > } > if self.defaults: # <- added > extra = {**self.defaults, **extra} <- added > if data: > s["headers"].extend( > [ > (b"content-length", str(len(data)).encode("ascii")), > (b"content-type", content_type.encode("ascii")), > ] > ) > s["_body_file"] = FakePayload(data) > ... > }}} > > **Fix Method 2: Modify ASGIRequest** > Alternatively, the ASGIRequest class can be adjusted to include default > headers in the META attribute during initialisation. > > {{{ > class ASGIRequest(HttpRequest): > > def __init__(self, scope, body_file): > self.scope = scope > ... > if isinstance(query_string, bytes): > query_string = query_string.decode() > self.META = { > **self.scope, <- # Added > "REQUEST_METHOD": self.method, > "QUERY_STRING": query_string, > "SCRIPT_NAME": self.script_name, > "PATH_INFO": self.path_info, > }}} > > This would be simliar to WSGIRequest, where self.META = environ is set > during the init phase. > However, it's worth noting that WSGI has a different META format. > > **asgi META ** > {{{ > {'asgi': {'version': '3.0'}, 'type': 'http', 'http_version': '1.1', > 'client': ['127.0.0.1', 0], 'server': ('127.0.0.1', '80'), 'scheme': > 'http', 'method': 'GET', 'headers': [(b'host', b'testserver'), > (b'cookie', b'')], 'HTTP_AUTHORIZATION': 'Bearer > eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE3MTQ0NDg2NzcsImV4cCI6MTcxNDQ1MjI3Nywic3ViX2lkIjoiMSIsInBsYXRmb3JtIjoiVk1TLXN0YWdpbmciLCJiYXNlX3VybCI6Imh0dHBzOi8vc3RhZ2luZy1jZW8tcG9ydGFsLWFwaS55b2dpeW8uY28ua3IvIiwicm9sZSI6InN0YWZmIiwidXNlcl9pZCI6IjEiLCJzdGFmZl9ncm91cF9pZCI6bnVsbH0 > .WWubd4iOUnsqbWO0ba-8mAsCCk3QDbBONvB_nznZQsk', 'path': > '/campaign/v1/campaigns/1/items/', 'query_string': > 'date_type=created_at&date_from=2024-04-28+03%3A44%3A37.484747%2B00%3A00&date_to=2024-05-02+03%3A44%3A37.484759%2B00%3A00', > 'REQUEST_METHOD': 'GET', 'QUERY_STRING': > 'date_type=created_at&date_from=2024-04-28+03%3A44%3A37.484747%2B00%3A00&date_to=2024-05-02+03%3A44%3A37.484759%2B00%3A00', > 'SCRIPT_NAME': '', 'PATH_INFO': '/campaign/v1/campaigns/1/items/', > 'wsgi.multithread': True, 'wsgi.multiprocess': True, 'REMOTE_ADDR': > '127.0.0.1', 'REMOTE_HOST': '127.0.0.1', 'REMOTE_PORT': 0, 'SERVER_NAME': > '127.0.0.1', 'SERVER_PORT': '80', 'HTTP_HOST': 'testserver', > 'HTTP_COOKIE': ''} > }}} > > ASGI META has separate 'headers' but the custom headers are not added > there. > > **wsgi META** > {{{ > {'HTTP_COOKIE': '', 'PATH_INFO': '/campaign/v1/campaigns/1/items/', > 'REMOTE_ADDR': '127.0.0.1', 'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', > 'SERVER_NAME': 'testserver', 'SERVER_PORT': '80', 'SERVER_PROTOCOL': > 'HTTP/1.1', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', > 'wsgi.input': <django.test.client.FakePayload object at 0x1061b0d90>, > 'wsgi.errors': <_io.BytesIO object at 0x104f42020>, 'wsgi.multiprocess': > True, 'wsgi.multithread': False, 'wsgi.run_once': False, > 'HTTP_AUTHORIZATION': 'Bearer > eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE3MTQ0NDg3OTQsImV4cCI6MTcxNDQ1MjM5NCwic3ViX2lkIjoiMSIsInBsYXRmb3JtIjoiVk1TLXN0YWdpbmciLCJiYXNlX3VybCI6Imh0dHBzOi8vc3RhZ2luZy1jZW8tcG9ydGFsLWFwaS55b2dpeW8uY28ua3IvIiwicm9sZSI6InN0YWZmIiwidXNlcl9pZCI6IjEiLCJzdGFmZl9ncm91cF9pZCI6bnVsbH0.MkbgS1zDaMLdDMLC0_Jpe_2O7VBtJD8km70Y0KlUb4g', > 'QUERY_STRING': > 'date_type=created_at&date_from=2024-04-28+03%3A46%3A35.172175%2B00%3A00&date_to=2024-05-02+03%3A46%3A35.172190%2B00%3A00'} > }}} > > Addressing this inconsistency will ensure that the behaviour of both > synchronous and asynchronous clients remains consistent and predictable > across Django applications. > > Thanks. New description: Currently, there is an inconsistency between Django's asynchronous AsyncClient and its synchronous counterpart Client regarding the handling of default headers. While the synchronous Client correctly includes default headers, the asynchronous AsyncClient ignores them. This behavior leads to discrepancies when utilizing fixtures with default headers, causing tests to fail unexpectedly. == Reproduction Steps: Set up a fixture with default headers for both synchronous and asynchronous clients. Utilize the fixtures in test cases and observe the behavior. Notice that the synchronous client includes default headers as expected, while the asynchronous client does not. == Code Snippets: {{{ @pytest.fixture(scope="session") def jwt_token(token_payload: dict[str, Any]) -> str: return jwt.encode({"abc", '"abc"}, key='123', algorithm="HS256") # this passes HTTP_AUTHORIZATION default header @pytest.fixture(scope="session") def sync_client_with_token(jwt_token) -> Generator[Client, None, None]: yield Client(HTTP_AUTHORIZATION=f"Bearer {jwt_token}") # this does not @pytest.fixture(scope="session") async def async_client_with_token(jwt_token) -> AsyncIterator[AsyncClient]: async_client = AsyncClient(HTTP_AUTHORIZATION=f"Bearer {jwt_token}") # async_client.defaults["AUTHORIZATION"] = f"Bearer {jwt_token}" yield async_client }}} AsyncRequestFactory.generic() does not currently check if self.defaults exists and ASGIRequest only check hard-coded header names in __init__() method, effectively ignoring rest of the self.scope values. Note that while RequestFactory.generic() method does not check whether self.defaults exist but WSGIRequest receives default values via ._base_environ() method when creating WSGIRequest instance. == Proposed Solutions: **Fix Method 1: Modify AsyncRequestFactory.generic() method** {{{ class AsyncRequestFactory(RequestFactory): def generic( self, method, path, data="", content_type="application/octet-stream", secure=False, *, headers=None, **extra, ): """Construct an arbitrary HTTP request.""" parsed = urlparse(str(path)) # path can be lazy. data = force_bytes(data, settings.DEFAULT_CHARSET) s = { "method": method, "path": self._get_path(parsed), "server": ("127.0.0.1", "443" if secure else "80"), "scheme": "https" if secure else "http", "headers": [(b"host", b"testserver")], } if self.defaults: # <- added extra = {**self.defaults, **extra} <- added if data: s["headers"].extend( [ (b"content-length", str(len(data)).encode("ascii")), (b"content-type", content_type.encode("ascii")), ] ) s["_body_file"] = FakePayload(data) ... }}} **Fix Method 2: Modify ASGIRequest** Alternatively, the ASGIRequest class can be adjusted to include default headers in the META attribute during initialisation. {{{ class ASGIRequest(HttpRequest): def __init__(self, scope, body_file): self.scope = scope ... if isinstance(query_string, bytes): query_string = query_string.decode() self.META = { **self.scope, <- # Added "REQUEST_METHOD": self.method, "QUERY_STRING": query_string, "SCRIPT_NAME": self.script_name, "PATH_INFO": self.path_info, }}} This would be simliar to WSGIRequest, where self.META = environ is set during the init phase. However, it's worth noting that WSGI has a different META format. **asgi META ** {{{ {'asgi': {'version': '3.0'}, 'type': 'http', 'http_version': '1.1', 'client': ['127.0.0.1', 0], 'server': ('127.0.0.1', '80'), 'scheme': 'http', 'method': 'GET', 'headers': [(b'host', b'testserver'), (b'cookie', b'')], 'HTTP_AUTHORIZATION': 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE3MTQ0NDg2NzcsImV4cCI6MTcxNDQ1MjI3Nywic3ViX2lkIjoiMSIsInBsYXRmb3JtIjoiVk1TLXN0YWdpbmciLCJiYXNlX3VybCI6Imh0dHBzOi8vc3RhZ2luZy1jZW8tcG9ydGFsLWFwaS55b2dpeW8uY28ua3IvIiwicm9sZSI6InN0YWZmIiwidXNlcl9pZCI6IjEiLCJzdGFmZl9ncm91cF9pZCI6bnVsbH0 .WWubd4iOUnsqbWO0ba-8mAsCCk3QDbBONvB_nznZQsk', 'path': '/campaign/v1/campaigns/1/items/', 'query_string': 'date_type=created_at&date_from=2024-04-28+03%3A44%3A37.484747%2B00%3A00&date_to=2024-05-02+03%3A44%3A37.484759%2B00%3A00', 'REQUEST_METHOD': 'GET', 'QUERY_STRING': 'date_type=created_at&date_from=2024-04-28+03%3A44%3A37.484747%2B00%3A00&date_to=2024-05-02+03%3A44%3A37.484759%2B00%3A00', 'SCRIPT_NAME': '', 'PATH_INFO': '/campaign/v1/campaigns/1/items/', 'wsgi.multithread': True, 'wsgi.multiprocess': True, 'REMOTE_ADDR': '127.0.0.1', 'REMOTE_HOST': '127.0.0.1', 'REMOTE_PORT': 0, 'SERVER_NAME': '127.0.0.1', 'SERVER_PORT': '80', 'HTTP_HOST': 'testserver', 'HTTP_COOKIE': ''} }}} ASGI META has separate 'headers' but the custom headers are not added there. **wsgi META** {{{ {'HTTP_COOKIE': '', 'PATH_INFO': '/campaign/v1/campaigns/1/items/', 'REMOTE_ADDR': '127.0.0.1', 'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'SERVER_NAME': 'testserver', 'SERVER_PORT': '80', 'SERVER_PROTOCOL': 'HTTP/1.1', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.input': <django.test.client.FakePayload object at 0x1061b0d90>, 'wsgi.errors': <_io.BytesIO object at 0x104f42020>, 'wsgi.multiprocess': True, 'wsgi.multithread': False, 'wsgi.run_once': False, 'HTTP_AUTHORIZATION': 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE3MTQ0NDg3OTQsImV4cCI6MTcxNDQ1MjM5NCwic3ViX2lkIjoiMSIsInBsYXRmb3JtIjoiVk1TLXN0YWdpbmciLCJiYXNlX3VybCI6Imh0dHBzOi8vc3RhZ2luZy1jZW8tcG9ydGFsLWFwaS55b2dpeW8uY28ua3IvIiwicm9sZSI6InN0YWZmIiwidXNlcl9pZCI6IjEiLCJzdGFmZl9ncm91cF9pZCI6bnVsbH0.MkbgS1zDaMLdDMLC0_Jpe_2O7VBtJD8km70Y0KlUb4g', 'QUERY_STRING': 'date_type=created_at&date_from=2024-04-28+03%3A46%3A35.172175%2B00%3A00&date_to=2024-05-02+03%3A46%3A35.172190%2B00%3A00'} }}} Addressing this inconsistency will ensure that the behaviour of both synchronous and asynchronous clients remains consistent and predictable across Django applications. Thanks. -- -- Ticket URL: <https://code.djangoproject.com/ticket/35414#comment:1> Django <https://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-updates+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/0107018f2d303a09-cb7d516c-235f-449f-87ab-457b02783cc0-000000%40eu-central-1.amazonses.com.