Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: StreamMultihasher interface #260

Open
Gozala opened this issue Jul 19, 2023 · 1 comment
Open

Proposal: StreamMultihasher interface #260

Gozala opened this issue Jul 19, 2023 · 1 comment

Comments

@Gozala
Copy link
Contributor

Gozala commented Jul 19, 2023

Rational

Current MultihashHasher API is really simple and a great fit for most cases, however it has certain drawbacks that make it a bad fit for more low-level use cases.

  1. API returns a Digest view for which you always have to allocate a buffer. If you have destination buffer for the multihash like a CAR or even a CID it creates unnecessary overhead of first allocating then copying the bytes into your destination.
    • It is also worth calling out that web crypto APIs require you to pass a ArrayBufferView.
  2. API assumes that you can hold complete payload in memory. When working on a large payloads this becomes a real problem. Unfortunately currently browsers do not provide streaming crypto APIs, but there seems to be some proposals to address this, never the less it would still be good to have it in node and non web native implementations.
  3. Sometimes you need a digest without the multiformat prefix and addressing above two without a way to omit the prefix would defeat the purpose in those instances.

Proposal

I end up defining separate interface to address above limitations and implemented two hashers that makes use of it

  1. blake3 hasher
  2. fr32-sha2-256-trunc254-padded-binary-tree hasher

I propose we uplift interface used by those implementations, possibly after iterating on it a bit here. I'm happy to update those implementations to align with interface that gets merged here.

@Gozala
Copy link
Contributor Author

Gozala commented Jul 19, 2023

Instead of writing up an interface here, I've created #261 so it's easier to provide feedback inline and iterate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant