The Zcash ArboretumFlyClient Guide PDF

2 The verification problem

This section fixes the object a light client verifies, the Zcash block header, and the baseline procedure that verifies a chain of headers in full. It states the trust model of the light-client protocol specified for Zcash and the problem the volume solves, with each term of its solution defined in one clause and constructed in a later section.

2.1 Header-only verification

Definition 2.1 (Light client).

A light client is a client that is not a full participant in the network of Zcash peers. It can send and receive payments, but does not store or validate a copy of the block chain (ZIP 221, “Terminology”).

The verifier of the FlyClient paper is a light client given the additional inputs fixed in Definition 4.16 (“Security and succinctness of chain proofs”, §4.4).

Byte strings and integers are related by the maps of the Ironwood Guide, §“Fields, groups, and encodings”, Definition “Integer and byte encodings”: the map 𝖨𝟤𝖫𝖤𝖮𝖲𝖯k sends an integer in {0,…,2k−1} to its ⌈k/8⌉-byte little-endian encoding, and the map 𝖫𝖤𝖮𝖲𝟤𝖨𝖯k sends a k/8-byte string to the integer it encodes in little-endian order.

Definition 2.2 (CompactSize).

The CompactSize encoding of an integer v∈{0,…,264−1} is the byte string

𝖢𝗈𝗆𝗉𝖺𝖼𝗍𝖲𝗂𝗓𝖾⁢(v):={𝖨𝟤𝖫𝖤𝖮𝖲𝖯8⁢(v)if ⁢0≤v≤252,𝟶⁢𝚡⁢𝙵⁢𝙳∥𝖨𝟤𝖫𝖤𝖮𝖲𝖯16⁢(v)if ⁢253≤v≤𝟶⁢𝚡⁢𝙵⁢𝙵⁢𝙵⁢𝙵,𝟶⁢𝚡⁢𝙵⁢𝙴∥𝖨𝟤𝖫𝖤𝖮𝖲𝖯32⁢(v)if ⁢𝟶⁢𝚡⁢𝟷𝟶𝟶𝟶𝟶≤v≤𝟶⁢𝚡⁢𝙵⁢𝙵⁢𝙵⁢𝙵⁢𝙵⁢𝙵⁢𝙵⁢𝙵,𝟶⁢𝚡⁢𝙵⁢𝙵∥𝖨𝟤𝖫𝖤𝖮𝖲𝖯64⁢(v)if ⁢𝟶⁢𝚡⁢𝟷𝟶𝟶𝟶𝟶𝟶𝟶𝟶𝟶≤v≤𝟶⁢𝚡⁢𝙵⁢𝙵⁢𝙵⁢𝙵⁢𝙵⁢𝙵⁢𝙵⁢𝙵⁢𝙵⁢𝙵⁢𝙵⁢𝙵⁢𝙵⁢𝙵⁢𝙵⁢𝙵,

of length 1, 3, 5 or 9 bytes respectively. Only this encoding, the shortest, is valid for v: a byte string that decodes to v under one of the other three prefixes MUST be rejected (ZIP 204, “CompactSize”).

ZIP 204 has status Draft; the minimal-encoding rule binds block headers independently, through the protocol specification, §“Block Header Encoding and Consensus”, which requires the header field 𝗌𝗈𝗅𝗎𝗍𝗂𝗈𝗇𝖲𝗂𝗓𝖾 to be encoded with the minimum number of bytes. That rule prevents a miner from testing several encodings of one Equihash solution against the difficulty filter.

The upgrade epochs delimited by the network upgrades of Definition 1.1 are fixed in “Upgrade epochs and epoch trees” (§6.2).

Definition 2.3 (Block header).

A Zcash block consists of a block header and a sequence of transactions. A block header hdr is the concatenation, in the following order, of the fields

Bytes Field Content
4 𝗇𝖵𝖾𝗋𝗌𝗂𝗈𝗇 signed 32-bit integer; the consensus rules require 𝗇𝖵𝖾𝗋𝗌𝗂𝗈𝗇≥4
32 𝗁𝖺𝗌𝗁𝖯𝗋𝖾𝗏𝖡𝗅𝗈𝖼𝗄 the block hash of the previous block
32 𝗁𝖺𝗌𝗁𝖬𝖾𝗋𝗄𝗅𝖾𝖱𝗈𝗈𝗍 the root of the transaction Merkle tree over the identifiers of the block’s transactions, stated in “The authorising-data root” (§6.5)
32 commitment field named as below
4 𝗇𝖳𝗂𝗆𝖾 unsigned 32-bit integer, the block timestamp
4 𝗇𝖡𝗂𝗍𝗌 unsigned 32-bit integer, an encoding of the target threshold
32 𝗇𝖭𝗈𝗇𝖼𝖾 32 arbitrary bytes
3 𝗌𝗈𝗅𝗎𝗍𝗂𝗈𝗇𝖲𝗂𝗓𝖾 𝖢𝗈𝗆𝗉𝖺𝖼𝗍𝖲𝗂𝗓𝖾⁢(1344)
1344 𝗌𝗈𝗅𝗎𝗍𝗂𝗈𝗇 an Equihash solution

with integers encoded little-endian. The commitment field is named 𝗁𝖺𝗌𝗁𝖱𝖾𝗌𝖾𝗋𝗏𝖾𝖽 before Sapling, 𝗁𝖺𝗌𝗁𝖥𝗂𝗇𝖺𝗅𝖲𝖺𝗉𝗅𝗂𝗇𝗀𝖱𝗈𝗈𝗍 in Sapling and Blossom, 𝗁𝖺𝗌𝗁𝖫𝗂𝗀𝗁𝗍𝖢𝗅𝗂𝖾𝗇𝗍𝖱𝗈𝗈𝗍 in Heartwood and Canopy, and 𝗁𝖺𝗌𝗁𝖡𝗅𝗈𝖼𝗄𝖢𝗈𝗆𝗆𝗂𝗍𝗆𝖾𝗇𝗍𝗌 from NU5; its value in each upgrade epoch is fixed in “The header consensus rules” (§6.7). The block hash of hdr is the 32-byte string

𝖡𝗅𝗈𝖼𝗄𝖧𝖺𝗌𝗁⁢(hdr):=SHA256d⁢(hdr),SHA256d⁢(x):=SHA256⁢(SHA256⁢(x)),

taken over the whole header serialisation, 𝗌𝗈𝗅𝗎𝗍𝗂𝗈𝗇𝖲𝗂𝗓𝖾 and 𝗌𝗈𝗅𝗎𝗍𝗂𝗈𝗇 included, in internal byte order, the order in which the hash function outputs its bytes. Let 𝖳𝗈𝖳𝖺𝗋𝗀𝖾𝗍:{0,…,232−1}→ℕ be the conversion of protocol specification, §“nBits conversion”. The header hdr satisfies proof of work when 𝗌𝗈𝗅𝗎𝗍𝗂𝗈𝗇 is a valid Equihash solution for hdr and

𝖫𝖤𝖮𝖲𝟤𝖨𝖯256⁢(𝖡𝗅𝗈𝖼𝗄𝖧𝖺𝗌𝗁⁢(hdr))≤𝖳𝗈𝖳𝖺𝗋𝗀𝖾𝗍⁢(𝗇𝖡𝗂𝗍𝗌)

(protocol specification, §“Difficulty filter”). The consensus rules further require, for a block at height η, that 𝗇𝖡𝗂𝗍𝗌=𝖳𝗁𝗋𝖾𝗌𝗁𝗈𝗅𝖽𝖡𝗂𝗍𝗌⁢(η), the value the difficulty-adjustment rule of protocol specification, §“Difficulty adjustment” assigns to that height.

The definition follows protocol specification, §“Block Header Encoding and Consensus” for the field table and its consensus rules, and ZIP 221, “Block header semantics and consensus rules” and ZIP 244 for the renaming of the commitment field. The hash SHA256d is registered in the Crypto Guide, §“The transparent layer’s hashes”. The Equihash condition is cited, not reconstructed: Consensus Guide, §“Equihash and the memoryless model”. The work of a block whose header carries 𝗇𝖡𝗂𝗍𝗌 is

𝗐𝗈𝗋𝗄⁢(hdr):=⌊2256𝖳𝗈𝖳𝖺𝗋𝗀𝖾𝗍⁢(𝗇𝖡𝗂𝗍𝗌)+1⌋,

and the total work ω⁢(C) of a chain C is the sum of the work of its blocks; a node considers the valid chain of greatest total work to be best, breaking ties between leaf blocks in favour of the block received first (Consensus Guide, §“The block tree and the most-work rule”, and its subsection §“Work, and the best chain”; protocol specification, §“The Block Chain” and §“Definition of Work”).

The fixed-width fields of Definition 2.3 occupy 4+32+32+32+4+4+32=140 bytes. Since 253≤1344≤𝟶⁢𝚡⁢𝙵⁢𝙵⁢𝙵⁢𝙵, the field 𝗌𝗈𝗅𝗎𝗍𝗂𝗈𝗇𝖲𝗂𝗓𝖾 occupies 3 bytes, and with the 1,344-byte solution a Zcash block header is 1,487 bytes, of which the solution is 1344/1487, about 90.4 per cent.

Definition 2.4 (Simplified payment verification).

A claimed chain of length N≥1 is a sequence of block headers C=(hdr0,…,hdrN−1) in which hdr0 is the header of the genesis block; header hdrη is the header at height η. Given one or more claimed chains, a simplified-payment-verification (SPV) client downloads every header of every claimed chain and checks, for each chain C:

  1. (S1)

    linkage: for 1≤η≤N−1, the field 𝗁𝖺𝗌𝗁𝖯𝗋𝖾𝗏𝖡𝗅𝗈𝖼𝗄 of hdrη equals 𝖡𝗅𝗈𝖼𝗄𝖧𝖺𝗌𝗁⁢(hdrη−1);

  2. (S2)

    proof of work: for 0≤η≤N−1, header hdrη satisfies proof of work in the sense of Definition 2.3, and its field 𝗇𝖡𝗂𝗍𝗌 equals 𝖳𝗁𝗋𝖾𝗌𝗁𝗈𝗅𝖽𝖡𝗂𝗍𝗌⁢(η), computed from hdr0,…,hdrη−1.

It rejects every chain that fails a check and accepts, among the rest, the chain C of greatest total work ω⁢(C), under the best-chain rule of the Consensus Guide, §“Work, and the best chain”.

The definition is that of ePrint 2019/226, Section 1, p. 2, instantiated with the header checks of protocol specification, §“Block Header Encoding and Consensus”. For a claimed chain of N blocks the client receives N headers, 1,487⁢N bytes, so its communication is Θ⁢(N). A block is at most 2,000,000 bytes (same section), so the saving over downloading full blocks is at most a constant factor, not an asymptotic one.

An SPV client checks neither the validity of transactions nor any consensus rule beyond those of the header, and therefore relies on the following assumption.

Assumption 2.5 (SPV assumption).

“The chain with the most PoW solutions follows the rules of the network and will eventually be accepted by the majority of miners.” (ePrint 2019/226, Section 1, p. 2, Assumption 1.)

The paper formalises the property an SPV client checks as Definition 4.14, and strengthens Assumption 2.5 quantitatively as Assumption 4.8 (“The (c,L)-adversary”, §4.3).

2.2 The light-client model of ZIP 307

ZIP 307 (Draft) specifies a light-client protocol in which a light client obtains, from a Zcash node and a proxy, compact blocks: per-block messages that may carry the block header. The trust model is stated in ZIP 307, “Security Model”: the node and proxy combination is assumed to be “honest but curious” and “is trusted to provide a correct view of the current best chain state and to faithfully transmit queries and responses”.

ZIP 307, “Block header validation” describes, as a proposed enhancement that the ZIP records as only partially implemented (only check (Z2) is made), checks of the SPV kind on each compact block that carries a header. With the fields named as in Definition 2.3, the checks are:

  1. (Z1)

    the field 𝗇𝖵𝖾𝗋𝗌𝗂𝗈𝗇 is at least the minimum block version;

  2. (Z2)

    the field 𝗁𝖺𝗌𝗁𝖯𝗋𝖾𝗏𝖡𝗅𝗈𝖼𝗄 equals the block hash of the previous compact block;

  3. (Z3)

    the Equihash solution is valid;

  4. (Z4)

    the target 𝖳𝗈𝖳𝖺𝗋𝗀𝖾𝗍⁢(𝗇𝖡𝗂𝗍𝗌) is non-zero and at most the proof-of-work limit 𝖯𝗈𝖶𝖫𝗂𝗆𝗂𝗍 (protocol specification, §“Constants”);

  5. (Z5)

    when the last 27 compact blocks all carry headers, the field 𝗇𝖡𝗂𝗍𝗌 is correct under the difficulty-adjustment rule;

  6. (Z6)

    the block hash, read as a little-endian integer, is at most 𝖳𝗈𝖳𝖺𝗋𝗀𝖾𝗍⁢(𝗇𝖡𝗂𝗍𝗌).

In that proposal, a compact block that fails a check MUST be discarded. The same section of ZIP 307 further describes a check that the commitment field equals the root of the Sapling note commitment tree recomputed from the compact block; from Heartwood that check no longer applies, since the field changes meaning (Definition 2.3; ZIP 221, “Block header semantics and consensus rules”). Checks (Z1) to (Z6), where made on every header, cost communication linear in the chain length N, as in Definition 2.4.

Of the trust that ZIP 307 places in the node and proxy, a FlyClient verifier would replace only the trust in which header chain carries the most work, under Assumption 4.8 (“The (c,L)-adversary”). It would replace the linear header checks by the logarithmic sampling of “The FlyClient protocol” (§5); it does not replace faithful transmission of queries and of compact-block contents. Such a verifier is designed but unspecified in the sense of Definition 1.3; its status is fixed in “A Zcash verifier” (§7.4).

2.3 Statement of the problem

The problem is the following. Under the SPV assumption (Assumption 2.5), strengthened quantitatively as Assumption 4.8 in “The (c,L)-adversary” (§4.3), a light client is to accept the head of the chain of greatest total work after communication sublinear in the chain length N. The inputs and the chain to be selected are those of Definition 2.4; the assumption is strengthened to Assumption 4.8, the selection holds except with a failure probability fixed in “The FlyClient protocol” (§5), and the communication falls from Θ⁢(N) to sublinear.

A client that does not check every proof of work “can be tricked into accepting an invalid chain by a malicious prover who can precompute a sufficiently-long chain using its limited computational power” (ePrint 2019/226, Section 1, p. 3); the adversary is fixed in “The FlyClient security model” (§4).

The solution uses four objects. A Merkle mountain range is a binary hash tree over a sequence that admits appending a leaf without changing its peaks or any node beneath them (“Merkle mountain ranges”, §3). A header chain commitment is a root, carried in every header after the genesis header, of such a range over all blocks strictly before that header (“Header chain commitments and per-sample checks”, §5.1). A work-weighted sample is a block drawn at a random point of cumulative work and checked against the commitment of the head (“The sampling distribution”, §5.4). A suffix of the chain is downloaded and checked in full, its extent fixed in the same subsection. FlyClient is the protocol that verifies a proof-of-work chain by drawing logarithmically many such samples and checking each against the commitment of the head (ePrint 2019/226, Section 1). The commitment of the paper covers the chain from the genesis block, whereas the commitment of ZIP 221 covers one upgrade epoch: the blocks from the activation height of the preceding network upgrade up to the committing block, exclusive (ZIP 221, “Specification”; “Upgrade epochs and epoch trees”, §6.2).