#35415: Adding content_type to StreamingHttpResponse on Linux causes memory
error
after streaming around 1GB-2GB of data.
-------------------------------+--------------------------------------
Reporter: LouisB12345 | Owner: nobody
Type: Bug | Status: closed
Component: HTTP handling | Version: 5.0
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
Changes (by Natalia Bidart):
* resolution: needsinfo => invalid
Comment:
Hello LouisB12345! Thank you for your report. As Sarah mentioned, the best
course of action at this point is to reach out to the community in the
[https://forum.djangoproject.com/c/internals/async/8 Django Forum (async
category)] to get help debugging your view, since we are not able to
reproduce. See below for the full details of the reproducer that I setup
locally, streaming a 3.3G iso image, without getting any memory usage
increase nor memory error.
Since the goal of this issue tracker is to track issues about Django
itself, and your issue seems, at first, to be located in your custom code,
I'll be closing this ticket as invalid following the
[https://docs.djangoproject.com/en/dev/internals/contributing/triaging-
tickets/#closing-tickets ticket triaging process]. If, after debugging,
you find out that this is indeed a bug in Django, please re-open with the
specific details and please be sure to include a small but complete Django
project to reproduce or a failing test case.
The reproducer I used looks as follows:
* A local Django project (`projectfromrepo`) with an app for this ticket
(`ticket_35415`)
* `uvicorn` installed and serving Django with `python -Wall -m uvicorn
projectfromrepo.asgi:application --reload`
* A views.py with this (slightly simplified) content:
{{{#!python
import aiofiles
import logging
import os
import threading
from django.http import StreamingHttpResponse
logger = logging.getLogger(__name__)
def debug(msg):
logger.info(msg)
print(msg)
def file_download(request):
file_name = "/home/nessita/debian-live-12.2.0-amd64-kde.iso"
assert os.path.exists(file_name)
debug(f"Requested {file_name} which stats {os.stat(file_name)=}.")
response = StreamingHttpResponse(
file_data(file_name), content_type="application/octet-stream"
)
response["Content-Disposition"] = f'attachment;
filename="{file_name}"'
return response
async def file_data(file_path, chunk_size=65536):
debug(f"Current threads are {threading.active_count()} opening file
{file_path}.")
async with aiofiles.open(file_path, mode="rb") as f:
teller = 0
while chunk := await f.read(chunk_size):
teller += 1
if teller % 1000 == 0:
debug(
f"Current threads are {threading.active_count()}
yielding chunk nr.{teller}."
)
yield chunk
}}}
* Included the following in the main urls.py `path("streaming/",
ticket_35415.views.file_download)`
* Visiting http://localhost:8000/streaming/ works without any issues and
the downloaded file matches the hash of the source file. What's printed in
the terminal:
{{{
(djangodev) [nessita@socrates projectfromrepo]$ python -Wall -m uvicorn
projectfromrepo.asgi:application --reload
INFO: Will watch for changes in these directories:
['/home/nessita/fellowship/projectfromrepo']
INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO: Started reloader process [435237] using StatReload
Requested /home/nessita/debian-live-12.2.0-amd64-kde.iso which stats
os.stat(file_name)=os.stat_result(st_mode=33188, st_ino=8093957,
st_dev=66306, st_nlink=1, st_uid=1001, st_gid=1001, st_size=3492741120,
st_atime=1698110826, st_mtime=1698112190, st_ctime=1698112191).
Current threads are 2 opening file /home/nessita/debian-
live-12.2.0-amd64-kde.iso.
Current threads are 4 yielding chunk nr.1000.
Current threads are 4 yielding chunk nr.2000.
Current threads are 4 yielding chunk nr.3000.
Current threads are 4 yielding chunk nr.4000.
Current threads are 4 yielding chunk nr.5000.
Current threads are 4 yielding chunk nr.6000.
Current threads are 4 yielding chunk nr.7000.
Current threads are 4 yielding chunk nr.8000.
Current threads are 4 yielding chunk nr.9000.
Current threads are 4 yielding chunk nr.10000.
Current threads are 4 yielding chunk nr.11000.
Current threads are 4 yielding chunk nr.12000.
Current threads are 4 yielding chunk nr.13000.
Current threads are 4 yielding chunk nr.14000.
Current threads are 4 yielding chunk nr.15000.
Current threads are 4 yielding chunk nr.16000.
Current threads are 4 yielding chunk nr.17000.
Current threads are 4 yielding chunk nr.18000.
Current threads are 4 yielding chunk nr.19000.
Current threads are 4 yielding chunk nr.20000.
Current threads are 4 yielding chunk nr.21000.
Current threads are 4 yielding chunk nr.22000.
Current threads are 4 yielding chunk nr.23000.
Current threads are 4 yielding chunk nr.24000.
Current threads are 4 yielding chunk nr.25000.
Current threads are 4 yielding chunk nr.26000.
Current threads are 4 yielding chunk nr.27000.
Current threads are 4 yielding chunk nr.28000.
Current threads are 4 yielding chunk nr.29000.
Current threads are 4 yielding chunk nr.30000.
Current threads are 4 yielding chunk nr.31000.
Current threads are 4 yielding chunk nr.32000.
Current threads are 4 yielding chunk nr.33000.
Current threads are 4 yielding chunk nr.34000.
Current threads are 4 yielding chunk nr.35000.
Current threads are 4 yielding chunk nr.36000.
Current threads are 4 yielding chunk nr.37000.
Current threads are 4 yielding chunk nr.38000.
Current threads are 4 yielding chunk nr.39000.
Current threads are 4 yielding chunk nr.40000.
Current threads are 4 yielding chunk nr.41000.
Current threads are 4 yielding chunk nr.42000.
Current threads are 4 yielding chunk nr.43000.
Current threads are 4 yielding chunk nr.44000.
Current threads are 4 yielding chunk nr.45000.
Current threads are 4 yielding chunk nr.46000.
Current threads are 4 yielding chunk nr.47000.
Current threads are 4 yielding chunk nr.48000.
Current threads are 4 yielding chunk nr.49000.
Current threads are 4 yielding chunk nr.50000.
Current threads are 4 yielding chunk nr.51000.
Current threads are 4 yielding chunk nr.52000.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35415#comment:4>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/0107018f398f6612-63d9420c-4715-47bb-8a4b-e63600fcc4c8-000000%40eu-central-1.amazonses.com.
