How to calculate the storage key for the STATE column family

[helmealex]

I would like to directly interrogate the node data from the RocksDb database. The data I would like to interrogate is the storage system events from the STATE column family. How is the key calculated for a given hash/block height, the System module and the events storage?

[bkchr]

Why do you want to read the raw storage? And not just use the provided methods?

[cheme]

To directly access data from the state column, you still need to use the

substrate/primitives/trie/src/lib.rs

Line 185 in 297a5e4

pub type TrieDB<'a, H> = super::TrieDB<'a, LayoutV1<H>>;

example of usage can be found in trie_backend_essence.rs from state machine crate.
When accessing directly data with rocksdb api, you will access 'key -> value' that are 'hash(encoded_trie_node)' -> 'value(encoded_trie_node)' (with a prefix in front of the hash for rocksdb, but prefix cannot be transformed to the actual storage key: it is just the begining of it).
So the fastest is certainly to build the triedb over your block root and the underlying database, then iterate from it.

[ggwpez]

Did you have a look at GSRPC for golang?
It is not completely up-to-date, but calculating storage keys can be done with it like here or here.

[helmealex]

So, from what I understand, you get the state root of the block, get the data from the sate column for that hash and the data is the encoded trie node for the root trie node. After that let's suppose I want to access the storage events. How? I attach the encoded keys for storage events to what key? Or i have to iterate through the child of the root trie node?

[helmealex]

Did you have a look at GSRPC for golang? It is not completely up-to-date, but calculating storage keys can be done with it like here or here.

I know, but these are the keys used to interrogate the RPC endpoint. You cannot use them to directly access the underlying rocksdb database.

[cheme]

If you want a storage event you need the key to the event value and access the stored event sfor this key (get) from the instantiated triedb struct (events being stored are those of the previous block).
In itself events are stored at

substrate/frame/system/src/lib.rs

Line 597 in 297a5e4

pub(super) type Events<T: Config> =

, and the trie key is calculated by the pallet storage macro macro (might be something like twox(name_of_pallet) ++ twox(name_of _field).
I know @shawntabrizi got some great javascrit to get the trie keys of the runtime storage, also found these post he writes which are quite nice https://www.shawntabrizi.com/substrate/querying-substrate-storage-via-rpc/ and https://www.shawntabrizi.com/substrate/transparent-keys-in-substrate/ (rpc for state are a good way to access storage, and basically if you want to go faster you can only just skip the rpc part and call directly the code in rust (calling the code behind the rpc directly may be rather good (the bigger cost is still accessing trie node from the db)).