decoding: fix panics by limiting max decode sizes in proofs, commitments and assets

[guggero]

Fixes #510.
Fixes #502.

Contains a commit (6174f9d) that is already in the v0-3-0-staging branch, but was incomplete, so I added itests for it as well.

This PR fixes a number of proof and asset decoding related issues discovered through fuzz tests. Both FuzzFile and FuzzProof were run for an hour each after the changes and they didn't discover any additional panics.

To help any reviewer, here are all the magic numbers in one place:

	// MaxAssetNameLength is the maximum byte length of an asset's name.
	// This byte length is equivalent to character count for single-byte
	// UTF-8 characters.
	MaxAssetNameLength = 64

	// MaxAssetEncodeSize is the size we expect an asset to not exceed in
	// its encoded form. This is used to prevent OOMs when decoding assets.
	// The main contributing factor to this size are the previous witnesses
	// which we currently allow to number up to 65k witnesses.
	MaxAssetEncodeSize = blockchain.MaxBlockWeight

	// FileMaxNumProofs is the maximum number of proofs we expect/allow to
	// be encoded within a single proof file. Given that there can only be
	// one transfer per block, this value would be enough to transfer an
	// asset every 10 minutes for 8 years straight. This limitation might be
	// lifted at some point when proofs can be compressed into a single
	// zero-knowledge proof.
	FileMaxNumProofs = 420000

	// FileMaxProofSize is the maximum size of a single proof in a proof
	// file. The maximum size of a meta reveal is 1 MB, so this value would
	// cap the number of additional inputs within a proof to roughly 128 of
	// assets with such large meta data.
	FileMaxProofSize = 128 * MetaDataMaxLen

	// FileMaxSize is the maximum size of a single proof file. This is not
	// just FileMaxNumProofs * FileMaxProofSize as only the minting proof
	// can commit to a large chunk of meta data. The other proofs are much
	// smaller, assuming they don't all have additional inputs. But we must
	// cap this value somewhere to avoid OOM attacks.
	FileMaxSize = 500 * 1024 * 1024

	// MetaDataMaxLen is the maximum length of the meta data. We limit this
	// to 1MB for now. This should be of sufficient size to commit to any
	// JSON data or even medium resolution images. If there is need to
	// commit to even more data, it would make sense to instead commit to an
	// annotated hash of the data instead. The reason for the limit is that
	// the meta data will be part of the genesis proof, which is stored in
	// the universe and needs to be validated by all senders and receivers
	// of the asset.
	MetaDataMaxLen = 1024 * 1024

	// MaxNumTaprootProofs is the maximum number of Taproot proofs there can
	// be in a proof. This limit represents the maximum block size in vBytes
	// divided by the size of a single P2TR output and is therefore only a
	// theoretical limit that can never be reached in practice.
	MaxNumTaprootProofs = blockchain.MaxBlockBaseSize / input.P2TRSize

	// MaxTaprootProofSize is the maximum size of a single Taproot proof.
	// A Taproot proof can contain a commitment proof which at maximum can
	// contain two MS-SMT proofs that max out at around 10k bytes each (in
	// the worst case).
	MaxTaprootProofSize = tlv.MaxRecordSize

	// MerkleProofMaxNodes is the maximum number of nodes a merkle proof can
	// contain. This is log2(max_num_txs_in_block) + 1, where max number of
	// transactions in a block is assumed to be 17k (theoretical value).
	MerkleProofMaxNodes = 15

We use math.MaxUint16 for the number of previous witnesses and the number of TX witness elements (as well as the max length for each individual witness stack element).

Everything else uses the tlv.MaxRecordSize (65k) size as that should fit all of those elements.

cc @Crypt-iQ

[Roasbeef]

I think we want to still allow the other type, so this way people can start to experiment with how they should be defined.

[ffranr]

FYI, with the code as is, adding a new meta reveal type would involve bumping the proof.Proof version.

[guggero]

Okay, I see the point about being able to add other types (without changing the proto or the proof version).
But then shouldn't we begin with a 1:1 mapping of the RPC enum to the native type so we don't need any switch statements in any place? So basically changing MetaOpaque MetaType = 1 to MetaOpaque MetaType = 0 so it matches META_TYPE_OPAQUE = 0;?

[Roasbeef]

FYI, with the code as is, adding a new meta reveal type would involve bumping the proof.Proof version.

Why's that the case? No interpretation is actually needed. You just need to re-create the TLV serialization then hash. In reality, the type here is more of a client side thing.

[guggero]

Yeah, he meant because of the marshal and un-marshal functions that returned an error for unknown meta types. The way I solved it now (direct cast, no switch statement) should allow users to use custom types.

[Roasbeef]

I think maybe we'll want a bit more breathing room here.

Mid-tern, we can also move to a route where we just keep them on disk and memory map, so then we never have to load the entire thing into user space memory at once.

[guggero]

Yeah, makes sense. Bumped it to 420000 (roughly 8 years).

[jharveyb]

I imagine we won't write proof files for each update in an LN channel right?

[guggero]

No, I don't think so. We'll just create a new single transition proof suffix for each update, and each update would be at the same "height" (same location in the proof file, so the file wouldn't be extended with every update).

[Roasbeef]

I imagine we won't write proof files for each update in an LN channel right?

They'll get replaced basically each time. Both sides also have everything they need to make the proof files that result from on-chain channel operations.

[jharveyb]

Hmm so if rewrite + update DB num proofs would be static 👌🏽

[ffranr]

Great to see these checks implemented!

[ffranr]

FYI, with the code as is, adding a new meta reveal type would involve bumping the proof.Proof version.

[ffranr]

I appreciate the small commits. I don't think this is very close to being ready. I just have some concerns with MB vs MiB, but that's about it.

[jharveyb]

Looking pretty good! I think some of the new limits are missing unit tests + this PR could also include similar mitigation for address decoding:

taproot-assets/address/address.go

Line 369 in bf4af96

return stream.Decode(r)

I think just using DecodeP2P here is good enough?

[guggero]

Looking pretty good! I think some of the new limits are missing unit tests + this PR could also include similar mitigation for address decoding:

taproot-assets/address/address.go

Line 369 in bf4af96

return stream.Decode(r)

I think just using DecodeP2P here is good enough?

Ah, good catch. Added!

[jharveyb]

Looking pretty good! Only large nit is to add test cases for the changes where we have custom limits vs. DecodeP2P(). So TxMerkleProofRecord with 16+ nodes, MetaRevealDataRecord, AnchorTxRecord, etc. Though the change is simple enough that maybe that's overkill and fuzzing time is good enough.

Ready to approve if so.

[jharveyb]

Looking really good! Only nit is now a separate proof size limit for RPC input. I think instead of adding that in decode it could actually be somewhere like proof.IsProofFile() as a guard before proofFile.Decode().

[ffranr]

I've reviewed the changes since my last review. All seem reasonable.

[jharveyb]

Discussed other places to limit proof sizes offline, these changes look good 🚀