feat: add support for batched JSON-RPC requests #65

lpahlavi · 2025-12-19T09:33:35Z

(DEFI-2565) Add support for JSON-RPC batch requests and responses. This mostly involves implementing existing JSON-RPC middlewares for JSON-RPC batches in addition to JSON-RPC single requests.

lpahlavi · 2026-01-06T16:03:10Z

@gregorydemay I did quite a bit of experimenting here with different solutions to make this as easy to use as possible despite all the generics involved with tower middlewares. For example I considered an enum wrapping single requests and batch requests, some traits for request/response pairs that are valid JSON-RPC calls (i.e. HttpJsonRpcRequest/HttpJsonRpcResponse and HttpBatchJsonRpcRequest/HttpBatchJsonRpcResponse), etc. In the end I think things as they are now are a pretty good tradeoff between being generic and usable, but let me know if you have any specific ideas/concerns and I can let you know why I didn't go in those directions.

lpahlavi · 2026-01-12T18:29:15Z

@gregorydemay One thing in particular I thought we could discuss: JSON-RPC batch responses are not necessarily in the same order as the requests. At some point, we will need to match responses to requests using IDs. I actually thought we could do this in the response filter where we check for consistent IDs. We could then return a vector of Results ordered like the requests, where each element is either the response for the request in the same index, or some sort of a MissingResponseId error.

Generally, I think in the end we might have per-response errors and overall errors for the batch so the return type from the Service for batch JSON-RPC should probably be something like Result<Vec<Result<Response, ErrorType1>, ErrorType2>. We would then have to think what to do in cases like: some IDs are unexpected in the response, but some responses are indeed there (we don't want to return an error for the batch as a whole, but what do we do with the unexpected responses?). Maybe a different struct to store the response of a batch JSON-RPC request would then make more sense? WDYT?

gregorydemay

Thanks a lot @lpahlavi for starting this effort!

gregorydemay · 2026-01-15T13:25:52Z

examples/json_rpc_canister/src/main.rs

+    // the current height of the Solana blockchain with different commitment requirements.
+    let requests = http::Request::post(solana_test_validator_base_url())
+        .header("Content-Type", "application/json")
+        .body(vec![


nice!

Maybe it would also be good to have an example with heterogeneous methods, where the batch request would contain requests for different methods?

Good idea! Done.

gregorydemay · 2026-01-15T13:28:53Z

examples/json_rpc_canister/src/main.rs

+        .map(|(response, expected_id)| {
+            let (id, result) = response.into_parts();
+            assert_eq!(id, expected_id.into());
+            result.expect("JSON-RPC API call should succeed")


nit: should we maybe print out the results to have something a bit more tangible when running the example?

I added some print statements, but AFAIK we can't see them since print statements from canisters are not show, right?

gregorydemay · 2026-01-15T13:30:47Z

examples/json_rpc_canister/src/main.rs

+        .collect()
+}
+
+fn batch_json_rpc_client<Params, Result>() -> impl Service<


understanding question: This feels very similar to json_rpc_client. Why are 2 clients needed? If a client can do JSON-RPC batch it could also do it with a single request, no?

Very good question! The function contents are the exact same, the only difference is the Service generic types. If I want this method to work for both single and batched requests, I need to make it generic and IMO it might end up being a bit more confusing:

fn client<Request, Response>( ) -> impl Service<http::Request<Request>, Response = http::Response<Response>, Error = BoxError> where Request: Debug + Serialize, Response: Debug + DeserializeOwned, CreateJsonRpcIdFilter<Request, Response>: CreateResponseFilter<http::Request<Request>, http::Response<Response>>, <CreateJsonRpcIdFilter<Request, Response> as CreateResponseFilter< http::Request<Request>, http::Response<Response>, >>::Error: Into<BoxError>, { ServiceBuilder::new() // Print request, response and errors to the console .layer(observability_layer()) // Convert request and response batches to JSON-RPC over HTTP and validate response IDs .layer(JsonRpcHttpLayer::new()) // Use cycles from the canister to pay for HTTPs outcalls .cycles_accounting(ChargeMyself::default()) // The actual client .service(Client::new_with_box_error()) }

WDYT?

gregorydemay · 2026-01-15T13:31:32Z

canhttp/src/http/json/tests.rs

-    use tower::{BoxError, Service, ServiceBuilder, ServiceExt};
+    use super::*;
+
+    // TODO: Add tests for batch JSON-RPC


TODO (this comment is to avoid forgetting about it)

gregorydemay · 2026-01-15T13:45:43Z

canhttp/src/http/json/mod.rs

+/// }
+/// ```
+///
+/// Create a simple batch JSON-RPC over HTTP client.


This seems very similar to the previous example and it's a priori difficult to see what the exact difference is. Maybe we could have a .batch method that transforms a service Service< HttpJsonRpcRequest<Params>, HttpJsonRpcResponse<Result>, Error > into an Service< HttpBatchJsonRpcRequest<Params>, HttpBatchJsonRpcResponse<Result>, Error >?

I don't think that would be possible because we would want to modify some of the internal middlewares within the service to achieve that. See this thread, but it's possible to build a generic method for the client that handles both types of requests.

gregorydemay · 2026-01-15T13:46:56Z

canhttp/src/http/json/mod.rs

-    Result: DeserializeOwned,
+    Request: Serialize,
+    Response: DeserializeOwned,
+    CreateJsonRpcIdFilter<Request, Response>:


understanding question: why do we need this new trait bound?

This is because the generic changed from just the contents of a JsonRpcRequest/JsonRpcResponse to now the contents of the http::Request/http::Response.

canhttp/src/http/json/response.rs

gregorydemay · 2026-01-15T13:54:44Z

canhttp/src/http/json/response.rs

 /// Ensure that the ID of the response is consistent with the one from the request
 /// that is stored internally.
-pub struct ConsistentJsonRpcIdFilter<O> {
+pub struct ConsistentJsonRpcIdFilter<Request, Response, Id> {


instead of being generic over Id wouldn't it be easier to make request_id a BTreeSet<Id>(in the case of non-batch it's then a singleton)?

I hesitated between the two, for me it doesn't make a big difference, but a singleton is also OK. The only thing is that there would have to be an assert when checking the response that the saved request ID is indeed a singleton when looking at a single JSON-RPC request ID.

I hesitated between the two, for me it doesn't make a big difference, but a singleton is also OK.

I think it's useful to avoid generics if we can since they spread a lot (need additional generic parameter everywhere).

assert when checking the response that the saved request ID is indeed a singleton when looking at a single JSON-RPC request ID.

I don't see why the logic for checking a single request ID should be different from that of checking a set.

I added a small refactoring to remove the generic and always use a Vec instead (since we now care about the request ordering).

gregorydemay · 2026-01-15T13:56:20Z

canhttp/src/http/json/response.rs

+            .body()
+            .iter()
+            .map(expected_response_id)
+            .collect::<BTreeSet<_>>();


Shouldn't there be an error if the given IDs are non-unique?

Good catch! I guess an assert here would not be ideal, though it could only be caused by an error in the client. WDYT? Should we add a fallible CreateResponseFilter trait so we can return an error when creating the filter (like TryCreateResponseFilter or something)?

I guess an assert here would not be ideal, though it could only be caused by an error in the client.

An error seems like it would be nicer, although we already have a panic when ensuring that the request IDs are not null.

gregorydemay · 2026-01-15T14:08:01Z

canhttp/src/http/json/response.rs

+            .body()
+            .iter()
+            .map(|response| response.id())
+            .collect::<BTreeSet<_>>();


This would not detect duplicate IDs in the response. I'm not sure I follow the logic with counting, I think it's not enough, one needs to identify the potential differences. How about something like

Request IDs are in a BTreeSet (by construction)

Collect response IDs in a vec.

Check cardinality of request IDs matches that of response IDs.

Kick out the request IDs that should be null according to the response

Collect responds IDs from vec to a BtreeSet, filtering out null elements.

Ensure both sets are equal.

WDYT?

That's true, overall though I think it might be actually better to return something like a vector of something like Result<JsonRpcResponse, NoResponseForThisIdError>. I think it would probably make sense to simply ignore responses with an unexpected ID (as we do in the single request case) and return a vector of responses that are ordered the same as the requests, based on corresponding IDs. WDYT? (See this comment.)

I had a look at making the return value of the service a Result<Vec<Result<JsonRpcResponse, _>>, _> but I changed my mind regarding this. The whole API becomes quite annoying to use (there are per-response errors and per-batch errors), only to handle the case where the server returns a batch with inconsistent IDs. In the end, I feel it's a good enough trade-off to just return an error for the whole batch in this case (i.e. inconsistent IDs in the batch leads to the whole batch response being an error).

However, I do feel that we should re-order the responses here to be in the same order as the requests, correlating with response IDs.

Now, concerning the null response IDs.

Kick out the request IDs that should be null according to the response

This, we can't do, since we can't correlate the responses with a null ID to a request ID in particular. Meaning we have to count the number of responses with null IDs and make sure this corresponds with the number of request IDs for which we don't have a corresponding response.

I had a closer look at the specs, and IIUC correctly, a null response ID should only occur with either a parse error, or an invalid request error (for which the error code and message are standardized in the specs). However, a parse error (i.e. the request is not valid JSON) should lead to a single JSON error object being returned (NOT an array). This means we can validate here that responses with a null ID should only be invalid request errors. We can then fill-in the response for all missing request IDs with the same invalid request error.

TL;DR The logic is the following:

Request IDs are in a vector (to record the request ordering)

Count the number of responses with null IDs and validate that they are "invalid request" errors

Store the remaining response IDs in a map,

Validate that there are no duplicate items (while building the map)

Correlate responses with request IDs

Validate that num(correlated responses) + num(responses with null IDs) = num(request IDs)

Construct a vector with the correlated responses and an invalid request error for request IDs with no corresponding error

WDYT?

Let's discuss the remaining open points online, I think it might be more efficient.

lpahlavi

Thanks a lot for the review and input @gregorydemay! I've left some comments and made some improvements to the examples. Let me know what you think!

lpahlavi · 2026-01-16T09:22:44Z

examples/json_rpc_canister/src/main.rs

+    // the current height of the Solana blockchain with different commitment requirements.
+    let requests = http::Request::post(solana_test_validator_base_url())
+        .header("Content-Type", "application/json")
+        .body(vec![


Good idea! Done.

lpahlavi · 2026-01-16T09:23:13Z

examples/json_rpc_canister/src/main.rs

+        .map(|(response, expected_id)| {
+            let (id, result) = response.into_parts();
+            assert_eq!(id, expected_id.into());
+            result.expect("JSON-RPC API call should succeed")


I added some print statements, but AFAIK we can't see them since print statements from canisters are not show, right?

lpahlavi · 2026-01-16T09:28:55Z

examples/json_rpc_canister/src/main.rs

+        .collect()
+}
+
+fn batch_json_rpc_client<Params, Result>() -> impl Service<


Very good question! The function contents are the exact same, the only difference is the Service generic types. If I want this method to work for both single and batched requests, I need to make it generic and IMO it might end up being a bit more confusing:

fn client<Request, Response>( ) -> impl Service<http::Request<Request>, Response = http::Response<Response>, Error = BoxError> where Request: Debug + Serialize, Response: Debug + DeserializeOwned, CreateJsonRpcIdFilter<Request, Response>: CreateResponseFilter<http::Request<Request>, http::Response<Response>>, <CreateJsonRpcIdFilter<Request, Response> as CreateResponseFilter< http::Request<Request>, http::Response<Response>, >>::Error: Into<BoxError>, { ServiceBuilder::new() // Print request, response and errors to the console .layer(observability_layer()) // Convert request and response batches to JSON-RPC over HTTP and validate response IDs .layer(JsonRpcHttpLayer::new()) // Use cycles from the canister to pay for HTTPs outcalls .cycles_accounting(ChargeMyself::default()) // The actual client .service(Client::new_with_box_error()) }

WDYT?

canhttp/src/http/json/response.rs

lpahlavi · 2026-01-16T09:31:29Z

canhttp/src/http/json/response.rs

+            .body()
+            .iter()
+            .map(expected_response_id)
+            .collect::<BTreeSet<_>>();


Good catch! I guess an assert here would not be ideal, though it could only be caused by an error in the client. WDYT? Should we add a fallible CreateResponseFilter trait so we can return an error when creating the filter (like TryCreateResponseFilter or something)?

lpahlavi · 2026-01-16T09:38:14Z

canhttp/src/http/json/response.rs

 /// Ensure that the ID of the response is consistent with the one from the request
 /// that is stored internally.
-pub struct ConsistentJsonRpcIdFilter<O> {
+pub struct ConsistentJsonRpcIdFilter<Request, Response, Id> {


I hesitated between the two, for me it doesn't make a big difference, but a singleton is also OK. The only thing is that there would have to be an assert when checking the response that the saved request ID is indeed a singleton when looking at a single JSON-RPC request ID.

lpahlavi · 2026-01-16T09:40:31Z

canhttp/src/http/json/response.rs

+            .body()
+            .iter()
+            .map(|response| response.id())
+            .collect::<BTreeSet<_>>();


That's true, overall though I think it might be actually better to return something like a vector of something like Result<JsonRpcResponse, NoResponseForThisIdError>. I think it would probably make sense to simply ignore responses with an unexpected ID (as we do in the single request case) and return a vector of responses that are ordered the same as the requests, based on corresponding IDs. WDYT? (See this comment.)

lpahlavi · 2026-01-16T09:43:25Z

canhttp/src/http/json/mod.rs

-    Result: DeserializeOwned,
+    Request: Serialize,
+    Response: DeserializeOwned,
+    CreateJsonRpcIdFilter<Request, Response>:


This is because the generic changed from just the contents of a JsonRpcRequest/JsonRpcResponse to now the contents of the http::Request/http::Response.

lpahlavi · 2026-01-16T09:44:01Z

canhttp/src/http/json/mod.rs

+/// }
+/// ```
+///
+/// Create a simple batch JSON-RPC over HTTP client.


I don't think that would be possible because we would want to modify some of the internal middlewares within the service to achieve that. See this thread, but it's possible to build a generic method for the client that handles both types of requests.

lpahlavi force-pushed the lpahlavi/defi-2565-batch-json-rpc branch 4 times, most recently from 8f0a69c to 0309f4f Compare December 19, 2025 12:25

DEFI-2565: Add support for batched JSON-RPC requests

0410582

lpahlavi force-pushed the lpahlavi/defi-2565-batch-json-rpc branch from 0309f4f to 0410582 Compare December 19, 2025 16:16

lpahlavi added 4 commits December 22, 2025 15:18

Add JsonRpcPayload trait

bcc612d

Refactor into JsonRpcCall trait

05829f5

Revert to generic types for Request and Response

dc4d0bb

Remove JsonRpcCall trait

dce5081

lpahlavi force-pushed the lpahlavi/defi-2565-batch-json-rpc branch from 8b24b98 to dce5081 Compare January 5, 2026 14:01

Cleanup example

e72299a

lpahlavi requested a review from gregorydemay January 6, 2026 16:03

lpahlavi marked this pull request as ready for review January 6, 2026 16:03

gregorydemay reviewed Jan 15, 2026

View reviewed changes

lpahlavi added 2 commits January 16, 2026 10:29

Make heterogeneous batch JSON-RPC request in example

b924dd3

Use Display in ConsistentResponseIdFilterError

5cf6e80

lpahlavi commented Jan 16, 2026

View reviewed changes

lpahlavi requested a review from gregorydemay January 16, 2026 09:46

lpahlavi added 3 commits January 16, 2026 16:05

Panic if request IDs are not unique

ec447ba

Make ConsistentJsonRpcIdFilter::request_ids always a BTreeSet

bc85a04

Correlate responses with request IDs in ConsistentJsonRpcIdFilter

af0a625

feat: add support for batched JSON-RPC requests #65

Are you sure you want to change the base?

feat: add support for batched JSON-RPC requests #65

Uh oh!

Conversation

lpahlavi commented Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lpahlavi commented Jan 6, 2026

Uh oh!

lpahlavi commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gregorydemay left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lpahlavi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

lpahlavi commented Dec 19, 2025 •

edited

Loading

lpahlavi commented Jan 12, 2026 •

edited

Loading