What flow-control of RPC requests mechanism(s) ?

[gww-parity]

Flow-control means things like max in-flight requests, dynamic control if there is capacity to receive one more request etc...

This can be achieved in to ways:

• in application -> e.g. based on amount of available memory or other metrics
• in proxy -> considering proxy (like Nginx) acting as rate limiter, application may need to inform/control rates on proxy according to it's capacity.

Were there discussions so far regarding requests flow-control and how to tackle it?

Each of approches (inside applicaiton and using external proxy server) have their pros and cons and have to be designed accordingly.

Instead of dynamic flow-control, we may be fine with fixed rate limiting (e.g. on Nginx) for now. Probably that's ok, as we may not need to head maximum utilisation (at least for now). Still, it would be nice to have captured informed decision making about topic.

It's RPC requests related, so it's kind of infrastructure question, therefore probably worth to threat in generalized way as problem is universal (every kind of request puts some vector of loads on service by principle).

[mxinden]

Were there discussions so far regarding requests flow-control and how to tackle it?

In case you are relating to the concrete Substrate RPC API, I am not the right person to reply, given that I have not been involved much in that part of the codebase.

In case you are referring to flow-control in general: Yes, there have been many discussions. Let me link a couple:

• sc_network::NetworkService::write_notifications should have proper back-pressure #5481
• client/network-gossip: Redesign validator #6335
• Add command-line flag to enable yamux flow control. #4892
• https://github.com/paritytech/substrate/issues/7750

Instead of dynamic flow-control, we may be fine with fixed rate limiting (e.g. on Nginx) for now.

See also w3f/polkadot-validator-setup#89 (comment).

Let me know if this is of some help.

[tomaka]

These links all apply to libp2p, while the opening post mentions the JSON-RPC server

[gww-parity]

To me all incoming requests generate load to server and maybe subject to integrate with one flow-control system.

[gww-parity]

Thank you! Pointers are helpful to start getting more context.

So basically to me it's this discussion is about generic question, i.e. about handling "all" incoming requests.

Therefore, also RPC API requests.

All in-flight requests generate load and should be taken into account when rate limiting.

[niklasad1]

//cc @tomusdrw

[tomusdrw]

The jsonrpc library that we use for RPC server implementation has no control flow implemented at all, afair it uses unbounded channels everywhere, so the requests are getting queued as far as the transport layer is able to receive them and dequeued as fast as we are able to process them.
There are two reasons why it was working like that:

1. All RPC requests were supposed to be lightweight.
2. RPC was never supposed to be exposed, and was only used by a single user on it's localhost (i.e. trusted user).

Both are no longer valid today, you can apply a band-aid in a form of rate-limiting in the reverse-proxy layer (nginx or jsonrpc-proxy), but this would have to be hardcoded and wouldn't be based on the actual processing time (rather on just fixed numbers acquired empirically).

AFAIR jsonrpsee library which is supposed to replace jsonrpc is much better in that regard, so it will be possible to back-pressure the transport layer and keep the load and memory within some limits.

[dvdplm]

All I can offer here is opinions. Like @gww-parity says in the ticket, there are pros and cons for both approaches. I think that using a proxy in front of the node is good enough and that the engineering effort required to ensure dynamic back pressure for all use-cases is better spent elsewhere. Until we hear of a concrete user requirement for dynamic rate limiting on the RPC layer my opinion is that we should encourage all and everyone to put a proxy in front of their node.
Even if we had high quality rate limiting implemented, If I were a node operator I'd put nginx in front of all of my nodes anyway. It's scary to me that there might be validators out there that don't do this.

[tomaka]

I think that using a proxy in front of the node is good enough

I don't think a reverse proxy offers any protection whatsoever here, unless you put absurdly low limits (like one connection at a time, max bandwidth of 10 bytes/sec).
A 100 bytes JSON-RPC request can trigger 5 to 10 seconds of intense I/O activity on the node, and the only thing a reverse proxy can limit is the number and sizes of the requests.

[dvdplm]

You can configure your proxy to limit things more accurately and ad-hoc for your situation, e.g. clamp down on metadata fetches but leave other things fairly open.

[dvdplm]

(You do have a point that selective rate limiting is tricky to do for the web socket case. Hmm.)

[gww-parity]

The other option is to make custom reverse-proxy (write or modify some existing code) that would dynamically adjust rates based on monitoring data from process or system/container.

Again, there are many approaches, all with their pros and cons. I am glad to collect all inputs and make some overview "where are we" and "where are we heading" , hopefully helpful in picking option reasonable engineering-work vs requirements wise.

[gww-parity]

P.S. if we would agree that this is in long term to be deprecated (i.e. public RPC requests) then solution could be also documenting things properly.

[gww-parity]

What reverse proxies offer limiting amount of in-flight connections / preferably with different limits for different request types?

(for Nginx e.g. I've found few parameters , but I am far from sysadmin ninja to know if it's answer to my question: multi_accept, limit_conn, worker_connections, worker_processes,
in https://geekflare.com/nginx-production-configuration/ and https://nginx.org/en/docs/ngx_core_module.html#worker_connections ; https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html . Eventually alternatives like https://www.sozu.io/ - Rust HTTP Reverse Proxy, "SŌZU receives and handles configuration changes at runtime and updates its internal configuration without restarts. You can update the configuration multiple times per second, and it will take care of lingering connections." )

[gww-parity]

Eventually maybe https://github.com/tomusdrw/jsonrpc-proxy this will be on use

Otherwise, after looking into it , it looks like configuring content awareness in Nginx is done via modules. Maybe we can use one of modules: https://www.nginx.com/resources/wiki/modules/ ; maybe Lua module https://www.nginx.com/resources/wiki/modules/lua/ would allow to configure such logic in Lua? (other task, but maybe helpful example https://stackoverflow.com/q/18877778 ) or another module https://github.com/calio/form-input-nginx-module .