ray.serve.grpc_util.gRPCInputStream#

class ray.serve.grpc_util.gRPCInputStream(request_iterator: AsyncIterator[T], *, cancel_event: Event | None = None)[source]#

Bases: AsyncIterator[T]

Async iterator wrapping an incoming gRPC request stream.

This class is used for client streaming and bidirectional streaming RPCs. It allows deployment methods to iterate over incoming request messages from the client.

Example usage in a deployment:

@serve.deployment
class BidiService:
    # Client streaming (stream-unary)
    async def ClientStreaming(self, request_stream: gRPCInputStream):
        total = 0
        async for request in request_stream:
            total += request.value
        return Response(result=total)

    # Bidirectional streaming (stream-stream)
    async def BidiStreaming(self, request_stream: gRPCInputStream):
        async for request in request_stream:
            yield Response(greeting=f"Hello {request.name}")

PublicAPI (beta): This API is in beta and may change before becoming stable.

Methods

`__init__`	Initialize the gRPCInputStream.
`cancel`	Mark the stream as cancelled.
`is_cancelled`	Check if the client has cancelled the stream.

Attributes

is_exhausted

Check if the stream has been fully consumed.