Deposit based storage incentivation for pallet_contracts

[athei]

Motivation

Up until recently pallet_contracts did prevent unbound state growth by charging rent or a deposit from the contract itself. This system was removed for various reasons: #9669. Read the linked PR description before engaging in this discussion. We want a replacement for that system. In the following I describe one idea at a high level and then go into detail for the various areas where questions might arise.

Overview

One rather obvious alternative to charging the contract for its own storage is to charge the caller that is responsible for creating this storage. This is distinct from and in addition to gas whose purpose it is to charge for execution time. Using ongoing payments (rent) is not really viable so it would be purely deposit based like in any other pallet: Calling a contract will transfer balance from the caller to the contract or vise versa depending on whether the call increased or decreased the storage usage. The balance is reserved in the contract's account so it cannot be used or moved away by the contract. The deposit made by the caller is calculated like this:

deposit = storage_used_before - storage_used_after

Note that deposit can be negative which constitutes a refund from the contract the caller who is removing this storage. This is not necessarily the original depositor.

One major criticism of this approach when compared with the contract based rent is that it makes contract providers inflexible with regard to their financing model because the storage is always payed by the caller. In theory, the old system allowed the contract authors to come up with their own financing model by for example pumping their own money into the contract to keep it afloat. In practice however, there are still gas costs that need to be payed by the caller so it wasn't never enough to allow fee less usage of contracts while making contract (language) development much harder.

I argue that the financing model of a contract should not live in the contract itself but should be provided by other means. This allows contract authors to concentrate on the business logic. Companies could provide proxy contracts to customers that are restricted in what they can do.

Implementation Details

This is a rather simple system from the distance but there are challenges to solve in some areas.

Contract Termination

A contract can decide to remove itself by calling the seal_terminate host function. As of right now a contract can call this function at any time in order to remove itself and all its associated storage. This function will continue to work in the same way with the little modification that all the deposits will be transferred to the beneficiary of the termination (as opposed to the caller).

tl;dr: Termination does not change its behaviour.

Code sharing

There are two kinds of storage whose size is controlled by users:

• The key value storage of a contract that can only be accessed by the contract it belongs to. This storage is created during contract execution.
• The contract code which is a wasm blob that is uploaded to chain using the instantiate_with_code extrinsic and can be shared between different contracts: A contract can be instantiated without uploading any new code to the chain. Instead, it can reference an existing code by hash.

The latter is what this section is about. The code can be shared between different contract instantiations. Questions arise around the removal of those contracts. We clearly want to allow the removal of those contracts for uploaders to regain their deposit. There are two distinct challenges with regard to code sharing:

Code blobs cannot be removed due to active users

We cannot remove code blobs which have contracts associated with them for obvious reasons. To enforce this invariant we have a reference count associated with each code blob. However a problem arises when 3rd parties start using a code blob uploaded by someone else. The uploader cannot delete the code to regain the deposit because of contracts it doesn't control.

The solution to this problem would be for contract authors to deny contract creation by entities they don't control. This can be done easily in the constructor of the contract. IMHO this is an elegant solution because it does not require baking in any logic into pallet_contracts. The drawback with this approach is that the default behavior would be to allow instantiation by anyone. However, that could be easily solved by contract languages that force authors to make an explicit decision in constructors. This would not require any changes whatsoever to the pallet_contracts.

tl;dr We do nothing and tell contract authors to be wary about who they allow to instantiate their code blob. Also we always refund the deposit to the original uploader and not to the remover (this is different from contract storage).

Race between upload and instantiation

Right now there is no way to "just upload" a code blob. You need to call instantiate_with_code which instantiates the first contract right away. The code blob is automatically deleted when the last contract that uses it is removed. This is an elegant solution because it does not require a separate extrinsic to remove an orphaned code blob.

However, there are requests for adding an extrinsic that can upload a code blob without an associated contract (patractlabs/redspot#136). That causes a race: Someone could delete the code hash in between the upload and instantiation. We could work around this with a minimum life time of a code blob but this will add new complexity.

tl;dr: As long as we don't add back put_code we don't need to do anything here. For now we don't. It can be added later, though.

User experience

With this change a user would pay for two distinct things when calling a contract:

• Gas (weight / execution time)
• Storage

The gas is is taken from the caller as transaction fee exactly like with any other transaction. The caller can limit the amount of gas that can be used by a call using a gas_limit.

For the storage a deposit is made to the contract and reserved there. Both values can be estimated by pre-running the call as RPC before submitting it as a transaction. It is called an estimation because between pre-running and transaction submission the state of the chain can change and with it the behavior of the call (this is why we have gas_limit).

In order to allow the same for storage we add a deposit_limit field which limits how much balance is allowed to be deposited as part of the call.

tl;dr We add a deposit_limit to the call arguments and a deposit_used to the call RPC. The latter one should be used by UIs together with gas_used to give the user a cost estimation. Note that deposit_used can be negative (refund).

[xlc]

We implemented a similar mechanism for Acala EVM.

Basically in additional to gas fee, caller also paying a storage deposit for every additional bytes wrote with this call. For every byte removed by this call, deposit is refunded to caller.

On user experience side, it just means some call will be expensive (e.g. transfer ERC20 token to a new account) and some will be cheap or even have some refund (e.g. transfer all remaining balance of a ERC20 to an existing account, hence wipe the balance storage).

There is a storage_limit parameter to limit maximum storage deposit can be charged by a transaction.
We do reference counting on contract code, but still charge the standard storage deposit for each contract instantiation just to keep thing simple.

Note that the storage deposit are reserved on the contract, not original caller. This will encourage people to call transactions that could potentially free up onchain storage for the storage deposit. It is up to the contract create to prevent such operation if it is a requirement. e.g. only the contract owner can remove the contract and hence getting the storage deposit back.

We are in a big refactor of Acala EVM but all the storage meter logic is already implemented and available here: https://github.com/AcalaNetwork/Acala/tree/master/modules/evm/src/runner

[atenjin]

I think this method is same as our experimental project https://github.com/patractlabs/patracts. This project is trying to modify pallet-contracts to implement the same thing. But we just stop this research now...

[athei]

Note that the storage deposit are reserved on the contract, not original caller. This will encourage people to call transactions that could potentially free up onchain storage for the storage deposit. It is up to the contract create to prevent such operation if it is a requirement. e.g. only the contract owner can remove the contract and hence getting the storage deposit back.

So when a caller creates storage the deposit is transferred to the contract and reserved there. This is an important distinction to what I am describing (balance reserved in the callers account). I like that idea because it is less complicated and does not require to store the original caller.

I skimmed through your code. I noticed that you don't store the original T::StorageDepositPerByte for each storage item. Won't that be a problem when this value changes with a runtime upgrade?

[xlc]

Yeah it is hard to handle the case when the deposit per byte changes.
e.g. The deposit was 1. a contract consumed 10 bytes and holds 10 deposits. An upgrade happened and the deposit is now 2. Additional 10 bytes is written and contract is new holding 30 deposits. Then 10 bytes was removed and should be refunded. Should it refund 10 or 20 or 15?

So I am deferring this question until we do need to adjust deposit per byte. We will need a migration but that's fine.

[cmichi]

I don't yet see the additional benefit this would bring over just a "simple" model of state expiry. Our current assumption is that a state expiry model will already result in limiting state bloat by quite a bit.

To me, the overhead of adding an economic incentive on top brings up a lot of questions and shifts some complexity back to the contract developers (because they should ideally provide an inverse to every "create" operation).

For the storage a deposit is made which is reserved from the callers account and unreserved when the storage is freed.

I like the model which @xlc described ‒ where the deposit is actually transferred to the contract ‒ better:

• I think it would be easier in terms of user experience. I can imagine that it will be irritating to explain why the wallet shows a certain amount as locked. This is something internal to the system and it should not be exposed to users that way. Also the tokens do not really belong to the user any longer, if the contract doesn't provide a way to free them they are gone forever. I can imagine cases where a deposit for a storage item might even be intended to never be freed up again.
• My understanding is that by having the deposit transferred to a contract, the logic for this would be kept within the contracts pallet, thus having a clearer boundary to other pallets.

We can store the creators address inline with the data (prepend it to the payload) so we don't generate extra storage accesses.

What happens if a field is mutated? Consider len: u32. Would the deposit be returned each time someone else is responsible for an increment of len?

We allow big payloads per storage item so that contracts are incentivized to "pack" structs and smaller collections into one storage item.

So if a user mutates just one particular field of e.g. a struct (or like you mentioned a small collection) the user would be responsible for depositing for the space of the entire storage item, right? Wouldn't the incentive then be to rather create as many fine-grained storage items as possible to have as little deposit necessary from a user as possible?

[athei]

I don't yet see the additional benefit this would bring over just a "simple" model of state expiry. Our current assumption is that a state expiry model will already result in limiting state bloat by quite a bit.

It could be that we want both because they operate on a completely different layer. State expiry merely mitigates the long term consequences of wasteful behavior without discouraging it. Whether state expiry will work out is not a clear cut and might be years away because it requires changes to the substrate storage layer. The proposal here can be implemented relatively quickly and can be removed if it turns out too be to big of a hurdle. I am not so sure that there will be acceptance to add this into an active ecosystem when state expiry won't fly.

To me, the overhead of adding an economic incentive on top brings up a lot of questions and shifts some complexity back to the contract developers (because they should ideally provide an inverse to every "create" operation).

My hunch is that it is reasonable to expect contract authors to allow for the removal of no longer used storage. For most contracts that should happen naturally. The problem with the old system was that it burdened beginners to deal with this stuff or their contracts could be evicted. A huge impediment to adoption. With this proposal this is more of an advanced topic that learners can ignore until they start optimizing for production use.

I like the model which @xlc described ‒ where the deposit is actually transferred to the contract ‒ better:

I agree. This also solves many of the open questions I had and it is way easier to implement.

So if a user mutates just one particular field of e.g. a struct (or like you mentioned a small collection) the user would be responsible for depositing for the space of the entire storage item, right?

No: deposit = size_after - size_before. Note that deposit could also be negative which constitutes a refund.

What happens if a field is mutated? Consider len: u32. Would the deposit be returned each time someone else is responsible for an increment of len?

No. Only the difference is refunded or deposited.

[athei]

Updated the top post to reflect the changes we discussed (deposit is transferred to the contract).

[athei]

Locked and converted to an issue. Continue discussion there: #9807