What are you missing in Substrate?

[shawntabrizi]

Thought I might kick off the discussion forum with an open ended call for people to speak about the features, functionalities, or whatever they feel is currently missing from Substrate.

Thanks!

[xlc]

Here is my Xmas wish list: https://github.com/paritytech/substrate/issues/created_by/xlc

[shawntabrizi]

Pick and post here your top 3 requests :)

[xlc]

I am going to be greedy.

As a Substrate builder:

• Exclusive reserves for pallet-balances #7223
  • Saves a lot of book-keeping code
• RPC to fetch all changed storages for a given block #4638
  • So we can build a database to easily identify the blocks that changes a particular storage key
• https://github.com/paritytech/substrate/issues/3187
  • So people can actually use child storage (without dealing with low level API)

As a Polkadot / Kusama user:

• https://github.com/paritytech/substrate/issues/7555
  • So I can know if my transaction is going to fail before submit it and able to tell what is the error. Or review the expected outcome before submit a transaction.
• https://github.com/paritytech/substrate/issues/7054
  • So I can rotate my proxy account without managing my transaction fee fund
• Allow council to slash treasury tip #7057
  • So we will have less low quality tips requests

[kaichaosun]

Message protocol for chat applications.

[burdges]

We do still want mixnets for off-chain messaging but doing polakdot correctly delayed that effort massively. We've some write ups like https://github.com/w3f/messaging/blob/master/incentives/incentives.tex around.

[olanod]

For Valiu this will start becoming a product requirement and the way we'll likely roll that I consider "good enough" is uing Matrix with the hopes it gets more decentralized over time like using libp2p as transport and having good Rust support to embedded in the node and compile to WASM to run in the browser. Also as a good enough start for identity I'd like to expand on their Identity Service API that comes with some key management endpoint definitions and that could also add blockchain addresses as another kind of 3PID that can map to the matrix id in a similar way it works for emails and phones.

[burdges]

1. Session backwards/full certificates grandpa: extend protocol for session key registration and usage #7398 - Eliminates almost all (current) slashing risks (in practice), but not soundness slashes. Also helps tell us when we're syncing and other nice things.

2. Audited (Timeout) Verification Functions (AVFs) - WASM modules invoked by PVFs that compute their runtime weight from their input and thus always return a deterministic weight irregardless of their running time. Auditing ensures they have sensible expected running time characteristics, which avoids timeout nastiness and gives us more flexibility when invoking native crypto from them, use native randomness, etc. Although I still need to write up more about these https://github.com/w3f/research-internal/issues/627 the idea would normally be that once invoked most return only valid or invalid, although more flexible options make sense sometimes.

Although I know the theory, I want folks to tell me..

3. How storage works and is used in substrate in practice - We need to design SNARK friendly storage, which means picking parameters like arity for the Poseidon hash function, and/or exposing these parameters to users.

4. When and how can host calls avoid serialization by passing &[T] or Vec<T> - We need this to expose fancy crypto in a nicer way. https://github.com/w3f/research-internal/issues/425 https://github.com/arkworks-rs/curves/issues/17

5. Impact of both 2, 3, and 4 on substrate's stability story

Alright one real issue and four future work discussion topics.. ;)

[bkchr]

2\. When and how can host calls avoid serialization by passing `&[T]` or `Vec<T>` - We need this to expose fancy crypto in a nicer way.  [w3f/research-internal#425](https://github.com/w3f/research-internal/issues/425)

If you pass raw bytes, they are not encoded. They are passed directly.

[burdges]

If I pass a &[u64] does it need to move it or can it just pass the pointer? If passing the pointer, are there alignment concerns that differ in WASM from native?

[bkchr]

Just raw bytes, &[u64] will be encoded.

[danforbes]

I am missing the faster, more responsive compile times of other languages/frameworks. Also the easier-to-read types/error messages.

[master-davidlee]

Official off-chain worker tutorial

[wpank]

#6271 - This would provide some nice monitoring signals : )

[wpank]

Also either #7398, context aware remote signing for slashable keys, or both

[Swader]

A battle tested, maintained CLI pallet manager that lets pallet makers define integration steps in a step by step way (i.e. the command line asks you about config values, possible depending pallets vs default/null values etc.) so Substrate users can just pull in the pallets they need.

$ spm add swader/aurapay --interactive

Looking for swader/aurapay on:
- crates.parity.io ... not found
- pallets.substrate.community ... found!

Installing swader/aurapay.

Missing dependencies: parity/aura.
Missing optional dependencies: parity/treasury.

Do you want to install the missing prerequisite parity/aura?
> Yes
Installing parity/aura...
Enter desired `slot_duration` (Determine the Aura slot-duration based on the Timestamp module configuration.) in seconds (value of 3 or more):
> 6
Enter desired `authorities` (The current authorities):

Do you want to install optional dependencies?
> No

Installing parity/aura with:
- slot_duration: 6
- authorities: null

Confirm y/n?
> y

......

parity/aura installed.

Installing swader/aurapay.
> Enter desired `block_reward`
10
> Skipping (missing optional dependency): Enter desired `treasury_allocation_percent` (Percent of block reward to give to treasury)

Install complete. Select:

1. Quit
2. Recompile
3. Recompile and bump the spec version (current 0.0.1)
> 1

This should be a funded, dedicated effort that has the potential to 10x the DX.

[jam10o-new]

would also be awesome to have this kind of thing include a a "runtime builder" that matches traits provided by pallets to traits required by other pallets.

[jam10o-new]

as much as possible, moving things like config, runtime apis, rpc, etc. out of node code and into the pallet crates that need/declare them - it's not "difficult" integrating a pallet into a node and having to dig through code I shouldn't have to write as a runtime/pallet dev (service, node), but I'd rather have all of the required changes be limited to the crate I am working on.

[drewstone]

Not sure if its mentioned but something that came to mind recently for me: compiling runtime modules as Ink! contracts or some deployable format for the contracts module.

Runtime module development is pretty nice and the development flow seems straightforward for me. Would be awesome if one didn't need to continue switching environments to development new apps, but rather new apps would be convertable into others.

[xlc]

I had the same request long time ago use-ink/ink#46

[Gauthamastro]

A node-template with BABE consensus :-)

[Gauthamastro]

@burdges Sure, but having node-template with different consensus options will make it easier for teams to build on the substrate. I had a hard time converting AURA to BABE for Polkadex.

[bkchr]

@Gauthamastro just check the normal substrate node.

[Gauthamastro]

@bkchr That's what I did in the end, but it was hard :-|.

[bkchr]

Yeah, we need better docs on this :)

[kaichaosun]

I made some effort here.