[WIP] Committee aggregation

docs/client/networking.md

@@ -33,6 +33,7 @@ Each node entry contains an ENR. This is an Ethereum Node Record. It includes:
 - The node's public key
 - Network address
 - Port numbers
+- Committee assignments (for aggregators)
 - Other metadata
 
 In production, dynamic discovery would replace static configuration.
@@ -62,15 +63,35 @@ Messages are organized by topic. Topic names follow a pattern that includes:
 
 This structure lets clients subscribe to relevant messages and ignore others.
 
+The payload carried in the gossipsub message is the SSZ-encoded, 
+Snappy-compressed message, which type is identified by the topic:
+
+| Topic Name                                                 | Message Type                | Encoding     |
+|------------------------------------------------------------|-----------------------------|--------------|
+| /lean/consensus/devnet3/blocks/ssz_snappy                  | SignedBlockWithAttestation  | SSZ + Snappy |
+| /lean/consensus/devnet3/attestations/ssz_snappy            | SignedAttestation           | SSZ + Snappy |
+| /lean/consensus/devnet3/attestation_{subnet_id}/ssz_snappy | SignedAttestation           | SSZ + Snappy |
+| /lean/consensus/devnet3/aggregation/ssz_snappy             | SignedAggregatedAttestation | SSZ + Snappy |
+
 ### Message Types
 
-Two main message types exist:
+Three main message types exist:
+
+* _Blocks_, defined by the `SignedBlockWithAttestation` type, are proposed by 
+validators and propagated on the block topic. Every node needs to see blocks 
+quickly.
 
-Blocks are proposed by validators. They propagate on the block topic. Every
-node needs to see blocks quickly.
+* _Attestations_, defined by the `SignedAttestation` type, come from all 
+validators. They propagate on the global attestation topic. Additionally,
+each committee has its own attestation topic. Validators publish to their
+committee's attestation topic and global attestation topic. Non-aggregating 
+validators subscribe only to the global attestation topic, while aggregators 
+subscribe to both the global and their committee's attestation topic.
 
-Attestations come from all validators. They propagate on the attestation topic. High volume
-but small messages.
+* _Committee aggregations_, defined by the `SignedAggregatedAttestation` type, 
+created by committee aggregators. These combine attestations from committee 
+members. Aggregations propagate on the aggregation topic to which every 
+validator subscribes.
 
 ### Encoding
 

docs/client/validator.md

@@ -2,8 +2,9 @@
 
 ## Overview
 
-Validators participate in consensus by proposing blocks and producing attestations. This
-document describes what honest validators do.
+Validators participate in consensus by proposing blocks and producing attestations. 
+Optionally validators can opt-in to behave as aggregators in a single or multiple 
+committees. This document describes what honest validators do.
 
 ## Validator Assignment
 
@@ -16,6 +17,32 @@ diversity helps test interoperability.
 In production, validator assignment will work differently. The current approach
 is temporary for devnet testing.
 
+## Attestation Committees and Subnets
+
+Attestation committee is a group of validators contributing to the common 
+aggregated attestations. Subnets are network channels dedicated to specific committees.
+
+In the devnet-3 design, however, there is one global subnet for signed 
+attestations propagation, in addition to publishing into per committee subnets.
+This is due to 3SF-mini consensus design, that requires 2/3+ of all 
+attestations to be observed by any validator to compute safe target correctly.
+
+Note that non-aggregating validators do not need to subscribe to committee
+attestation subnets. They only need to subscribe to the global attestation 
+subnet.
+
+Every validator is assigned to a single committee. Number of committees is 
+defined in config.yaml. Each committee maps to a subnet ID. Validator's
+subnet ID is derived using their validator index modulo number of committees.
+This is to simplify debugging and testing. In the future, validator's subnet ID
+will be assigned randomly per epoch.
+
+## Aggregator assignment
+
+Some validators are self-assigned as aggregators. Aggregators collect and combine
+attestations from other validators in their committee. To become an aggregator,
+a validator sets `is_aggregator` flag to true as ENR record field.
+
 ## Proposing Blocks
 
 Each slot has exactly one designated proposer. The proposer is determined by
@@ -52,7 +79,7 @@ receive and validate it.
 
 ## Attesting
 
-Every validator attestations in every slot. Attesting happens in the second interval,
+Every validator attests in every slot. Attesting happens in the second interval,
 after proposals are made.
 
 ### What to Attest For
@@ -78,8 +105,8 @@ compute the head.
 
 ### Broadcasting Attestations
 
-Validators sign their attestations and broadcast them. The network uses a single topic
-for all attestations. No subnets or committees in the current design.
+Validators sign their attestations and broadcast them into the global 
+attestation topic and its corresponding subnet topic.
 
 ## Timing
 
@@ -98,11 +125,7 @@ blocks and attestations.
 Attestation aggregation combines multiple attestations into one. This saves bandwidth and
 block space.
 
-Devnet 0 has no aggregation. Each attestation is separate. Future devnets will add
-aggregation.
-
-When aggregation is added, aggregators will collect attestations and combine them.
-Aggregated attestations will be broadcast separately.
+Devnet-3 introduces signatures aggregation. Aggregators will collect attestations and combine them. Aggregated attestations will be broadcast separately.
 
 ## Signature Handling
 

src/lean_spec/subspecs/chain/config.py

@@ -37,6 +37,12 @@
 VALIDATOR_REGISTRY_LIMIT: Final = Uint64(2**12)
 """The maximum number of validators that can be in the registry."""
 
+ATTESTATION_COMMITTEE_COUNT: Final = Uint64(1)
+"""The number of attestation committees per slot."""
+
+COMMITTEE_SIGNATURE_THRESHOLD_RATIO: Final = 0.9
+"""Default ratio of committee signature participation required to trigger aggregation."""
+
 
 class _ChainConfig(StrictBaseModel):
     """
@@ -52,11 +58,19 @@ class _ChainConfig(StrictBaseModel):
     historical_roots_limit: Uint64
     validator_registry_limit: Uint64
 
+    # Attestation / Networking
+    attestation_committee_count: Uint64
+
+    # Aggregation behavior
+    committee_signature_threshold_ratio: float
+
 
 # The Devnet Chain Configuration.
 DEVNET_CONFIG: Final = _ChainConfig(
     seconds_per_slot=SECONDS_PER_SLOT,
     justification_lookback_slots=JUSTIFICATION_LOOKBACK_SLOTS,
     historical_roots_limit=HISTORICAL_ROOTS_LIMIT,
     validator_registry_limit=VALIDATOR_REGISTRY_LIMIT,
+    attestation_committee_count=ATTESTATION_COMMITTEE_COUNT,
+    committee_signature_threshold_ratio=COMMITTEE_SIGNATURE_THRESHOLD_RATIO,
 )

src/lean_spec/subspecs/containers/attestation/attestation.py

@@ -20,6 +20,7 @@
 from lean_spec.subspecs.ssz import hash_tree_root
 from lean_spec.types import Bytes32, Container, Uint64
 
+from ...xmss.aggregation import AggregatedSignatureProof
 from ...xmss.containers import Signature
 from ..checkpoint import Checkpoint
 from .aggregation_bits import AggregationBits
@@ -107,3 +108,10 @@ def aggregate_by_data(
             )
             for data, validator_ids in data_to_validator_ids.items()
         ]
+
+class SignedAggregatedAttestation(Container):
+    data: AttestationData
+    """Combined attestation data similar to the beacon chain format."""
+
+    proof: AggregatedSignatureProof
+    """Aggregated signature proof covering all participating validators."""