A RuntimeError: Stream consumed error is raised when request.body() is read in the custom middleware

[theredfoxlee]

Hi,

I encountered the following issue while using Starlette today :)

Issue description

A RuntimeError: Stream consumed error is raised when request.body() is read in the custom middleware (1) after call_next, given that a route uses request.body().

How to reproduce the issue?

Code

from starlette.applications import Starlette
from starlette.responses import JSONResponse
from starlette.routing import Route
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.middleware import Middleware

async def testpage(request):
    await request.body()
    return JSONResponse({'hello': 'world'})


class CustomMiddleware(BaseHTTPMiddleware):

    async def dispatch(self, request, call_next):
        response = await call_next(request)
        await request.body()  # THIS WILL RAISE -> RuntimeError: Stream consumed
        return response


app = Starlette(
    debug=True,
    routes=[Route('/', testpage, methods=["PUT"])],
    middleware=[Middleware(CustomMiddleware)]
)

Dependencies

starlette==0.37.2
uvicorn==0.29.0

Commands

• 1st terminal: uvicorn example:app
• 2nd terminal: curl -XPUT http://127.0.0.1:8000 -d '123'

Known workarounds

1. Use request.body() at least once before call_next to ensure that the _body is cached.

I prefer not to use a workaround, as I only want to read the body in the middleware conditionally, based on the response, rather than always.

---

This is my first time using starlette. Is this usage incorrect, or is it a bug?

[theredfoxlee]

I believe it's a bug. May I move it to the issues?

[theredfoxlee]

@lukasab, thank you for your reply. However, I believe this situation is different as there is no exception in the simple middleware I presented above, so there's no reason to check whether the request is disconnected or not. The problem here is that Starlette does not allow reading the request.body in the middleware after the route has read the request.body. On the other hand, it does allow for reading it before the route reads the request body. Is this expected behavior? I have not seen it documented anywhere. I would expect that the body would be cached once read by the route.

[theredfoxlee]

@Kludex, isn't the reproducer posted above enough to state that it's a bug or misuse? The situation occurs in 100% of situations

[lukasab]

I don't get it. Isn't your middleware raising the Runtime exception? This means that during the middleware the client has disconnected which internally raises the RuntimeError with "Stream consumed" when you try to read the request body. Would this work for you?

class CustomMiddleware(BaseHTTPMiddleware):

    async def dispatch(self, request, call_next):
        response = await call_next(request)
        if await request.is_disconnected():
            return Response(status_code=HTTPStatus.NO_CONTENT)
        await request.body() # should not raise the error anymore
        return response

[theredfoxlee]

Isn't your middleware raising the Runtime exception?

It is.

This means that during the middleware the client has disconnected

Sadly, it doesn't always mean that.

await request.is_disconnected() returns False in this situation, which means that the client is connected and request.body() still raises the exception despite of it (due to its internal implementation issues).

Here's the same exception with your snippet: log.txt.

---

Once again, I think it's a bug. Starlette already implemented a workaround for reading the body before call_next last year: #1692; what's left is handling the same situation after call_next.

[theredfoxlee]

The issue here is that we're using starlette.requests.Request inside the route, which seems not to propagate the read .body to starlette.middleware.base._CachedRequest used in the middleware.