WebSockets?

[sowelipililimute]

Is your feature request related to a problem? Please describe.
The long incomplete HTTP request/response (and HTTP2 requirement for bidirectional streaming) used by the current streams implementation was already found to be unsuitable for Harmony due to the same issues outlined in the documentation; the streams implementation is blocking us from adopting Buf Connect as a more standard replacement to our home-grown hRPC.

Describe the solution you'd like
Support for WebSockets for streams to play nicer with proxies and other standard tooling

[akshayjshah]

Hi @pontaoski! We've discussed this extensively internally. Websockets are definitely better supported in some environments, but they have their own drawbacks too. We'll have a more detailed proposal (with some pros and cons) up in a few weeks for community discussion.

[mattrobenolt]

I'd imagine something like this could be entirely implemented as a middleware? Seems likely it could even be an entirely separate library.

Obviously this depends on what the actual transport codec ends up looking like, but would probably work similar to how gRPC-web works in a traditional gRPC environment.

[akshayjshah]

@mattrobenolt - for sure. Middleware is most attractive if we want this to be an optional add-on that's mostly for browser-to-backend communication. Having it built-in is maybe better (if it's possible with the current interfaces) if we want it to be first-class, even for backend-to-backend communication.

All things to discuss in the proposal!

[tw1nk]

Did anything happen with this? What about the proposal?

[kareemgithubprotonme]

Any details on implementing middleware to serve WebSockets would be appreciated

[chrispine]

Next step here would be for someone to come up with a proposal / RFC, ideally with POC client and server implementations.

[elee1766]

we are currently using an inhouse protocol which is basically jsonrpc2 over websocket described by openapi, and would love to be able to switch to connect rpc. we make many requests from the frontend and websockets are more performant, so that is why we are doing what we do.

maybe something is better than nothing, so i will post some musings and maybe this can be the start of something, since it seems like there is interest but nobody has really said anything concrete. lease give any input/feedback, this is just me posting some thoughts, and maybe we can evolve this to a proof of concept (i am comfortable with go, js)

challenges

the obvious challenge when working with a websocket is that your responses are now asynchronous, and the client doesn't have basically, the server telling it when its request is done. you lose both the guarantee of a response, along with losing free tracking for your response.

this means that you need your own mechanism for tracking the response to your request. this also means that timeouts must be implemented in client business logic instead of being handled by say, http. this complicates things because it means that clients must have state, and basically require some sort of asynchronous runtime to handle efficiently, implementation wise.

a second challenge with websockets is stream congestion. an aggressive application or stream may pollute an entire websocket connection (say, internal applications exchanging data at high speeds), causing requests to time out. it is arguable what connect should do in this case, if connect clients should provide multiplexed websockets, fallback to http, or just do nothing (users will see requests error with "no response in time", and maybe it can be suggested to use a different transport)

a third challenge with websocket is that you cannot defer the sending of a request. If you start writing frames for your 1gb message, your choices are either to 1. finish sending that message, or 2. close the entire connection (therefore cancelling all pending requests). no other request can write frames until you are done with your message.

possible solution - using a single websocket for multiple request/stream using id (like jsonrpc)

unary rpcs

websockets have a concept of a "message". a message can be basically any size (2^63), and can contain any sort of information. messages are the boundaries for compression, if enabled, and like said above, are delivered in continuous frames. we can use the websocket in binary mode to make it so that we can send compressed and protobuf encoded data

so a handler for websocket server can read in from its connection one message at a time, knowing the exact start and stop boundaries at each step.

because of this, I think that it makes most sense to actually just send an extension of the the entire existing connect protocol unary-request into the message. In fact, it's probably find to even keep the HTTP prefix (but with no Host, since that is not important), but staying HTTP/1.1 compatible makes parsing easier.

i think we can borrow the request-id concept from jsonrpc, and put it as a required header. let's call it Unary-Request-Callback-Id, and force it to be a uuid. this will make it easier for systems to route the workload.

so modifying the examples from the docs:

1> POST /connectrpc.greet.v1.GreetService/Greet HTTP/1.1
1> Unary-Request-Callback-Id: 1b057094-cc20-4abb-ba81-6b335d078775
1> Content-Type: application/json
1>
1> {"name": "Buf"}


<1 HTTP/1.1 200 OK
<1 Content-Type: application/json
<1 Unary-Request-Callback-Id: 1b057094-cc20-4abb-ba81-6b335d078775
<1
<1 {"greeting": "Hello, Buf!"}

this gives us the ability to send errors therefore in the same format.

1> POST /connectrpc.greet.v1.GreetService/Greet HTTP/1.1
1> Unary-Request-Callback-Id: 1b057094-cc20-4abb-ba81-6b335d078775
1> Content-Type: application/proto
1>
1> <uncompressed binary Protobuf>

<1 HTTP/1.1 501 Not Implemented
<1 Unary-Request-Callback-Id: 1b057094-cc20-4abb-ba81-6b335d078775
<1 Content-Type: application/json
<1
<1 {
<1   "code": "unimplemented",
<1   "message": "connectrpc.greet.v1.GreetService/Greet is not implemented"
<1 }

If the client doesn't get a response from the server, it should return a synthetic 504 code error to the caller.

streaming rpcs

I think it's important to support bidi streams over websocket (even though grpc has no plans to).

To do this though, we will have to cross message boundaries, because of challenge 3 described above. However, to do this, we need to be efficient, since we don't want to send a bunch of extra header information every time.

so to do this, i think that there needs to be some sort of mini-handshake to initiate a stream. let's just use the same sort of uuid header assigned by client, call it Streaming-Stream-Id, with server responsible to error on uuid reuse

we can save some bytes per message by encoding the UUID as the path for future sends of data. we can do the same by replacing the OK with the streaming ID as well. the http 1.1 standard says that reason phrases MAY be replaced by local equivalents without affecting the protocol, which means that parsers should all be able to parse it correctly.

we should allow sending data with the initial request, to save on even more bytes.

we can make content type only required if there is data, because we can determine via whether or not the 200 reason is OK or a /UUID, and if it's a /UUID, then we know it is a stream

the server SHOULD respond a 200 as soon as it receives a streaming request, or a not 200 if not supported, even if it has nothing to send.

the client SHOULD NOT send more data to the stream until the server has returned an initial ack.

so it would look something like

1> POST /connectrpc.greet.v1.GreetService/GreetGroup HTTP/1.1
1> Streaming-Stream-Id: 95ad7121-103d-4f50-9e56-bdccfcc3b909
1> Content-Type: application/connect+json
1>
1> <flags: 0><length: 15>{"name": "Buf"}

<1 HTTP/1.1 200 /95ad7121-103d-4f50-9e56-bdccfcc3b909

2> PATCH /95ad7121-103d-4f50-9e56-bdccfcc3b909 HTTP/1.1
2> Content-Type: application/connect+json
2> <flags: 0><length: 19>{"name": "Connect"}

<2 HTTP/1.1 200 /95ad7121-103d-4f50-9e56-bdccfcc3b909
<2 Content-Type: application/connect+json
<2 <flags: 0><length: 39>{"greeting": "Hello, Buf and Connect!"}
<2 <flags: 2><length: 2>{}

streams likely need to also send some extra parametsr on first request, most importantly, the max idle timeout. this is because otherwise, streams can forget to close themselves and ddos the server.

other things to think about

Maybe should use a uint64/int64 instead of a uuid for stream/request id?

Should you be able to send headers in your initial websocket dial, to set persistent headers for the session? Should these be a separate set of headers, that maybe can set defaults that can be overridden per request? (this could maybe save a lot of bytes, for instance, share the content type)

Should we consider putting the request uuid/id in the request path, to save the bytes of the header name?

is it really worth saving the bytes on the streaming side to make the protocol more efficient, at the cost of breaking compatibility with the connect rpc over http protocol?

should websocket compression (permessage-deflate) be supported?

what happens when the server dies? what should be communicated to the client?

what happens if the server fails over? what should be communicated to the client?

I dont know, im sure there are many things i have not thought of.