diff --git a/docs/adr/0005-signing-service-and-transparency-log-api.md b/docs/adr/0005-signing-service-and-transparency-log-api.md new file mode 100644 index 0000000..2999bb0 --- /dev/null +++ b/docs/adr/0005-signing-service-and-transparency-log-api.md @@ -0,0 +1,171 @@ +# ADR-0005: Signing-Service- und Transparency-Log-API-Vertrag + +- **Status:** accepted +- **Datum:** 2026-10-25 +- **Entscheider:** CTO +- **Betroffene Dokumente:** `docs/architecture/v1.1.md` §4.2 (DSSE), §4.5a (Query Attestation), §8 (Transparency Log), `services/admin-api/internal/attest/`, `services/admin-api/internal/tlog/` + +## Kontext und Problemstellung + +Jedes Mal, wenn ein ProofedMesh-Dienst eine Attestation signieren will (Approval, Transformation, Query, Rate-Change, Receipt-Ingest, SAP-Buchungs-Nachweis u.a.), muss er dafür einen zentralen Dienst aufrufen und das Ergebnis anschließend in den Transparency Log eintragen. Im Trust-Core (`trust-api`, `signing-service`, `trillian-log`) liegen die kryptografischen Schlüssel und die Append-Only-Historie; die aufrufenden Dienste (admin-api, agent-travel, agent-sink-posting, …) kennen weder Schlüssel noch Log-Interna. + +Der Vertrag zwischen "aufrufender Service" und diesen beiden zentralen Komponenten ist genau einmal zu entscheiden und dann für alle Services identisch — sonst entstehen pro Konsument eigene Sprachen und wir verlieren die Austauschbarkeit (Policy: Signing Service kapselt OpenBao-Transit, Transparency Log kapselt Trillian). + +## Entscheidung + +Zwei kleine, deliberat flach gehaltene REST-APIs. Keine gRPC-Pflicht, kein Protobuf-Zwang. Transportsicherheit per mTLS zwischen Dienst-Namespaces. Authentisierung per **Pocket-ID-Access-Token** (`client_credentials`, Workload-Identität) plus Standard-OIDC-Discovery. + +### Signing Service — `POST /api/v1/sign/dsse` + +``` +POST https://trust.basicstack.de/api/v1/sign/dsse +Authorization: Bearer +Content-Type: application/json +``` + +Request: +```json +{ + "key_ref": "trust-api/rate-change-signer-2026-q4", + "statement": { + "_type": "https://in-toto.io/Statement/v1", + "subject": [{"name": "...", "digest": {"sha256": "..."}}], + "predicateType": "https://basicstack.de/attestations//v1", + "predicate": { ... } + } +} +``` + +- `key_ref` referenziert einen in OpenBao Transit registrierten Schlüssel. Der Signing Service prüft via OPA, dass der aufrufende `client_id` berechtigt ist, mit diesem `key_ref` zu signieren (siehe `policies/signing/*.rego`, später). +- `statement` ist ein vollständiges in-toto-v1-Statement. + +Response `200`: +```json +{ + "envelope": { + "payloadType": "application/vnd.in-toto+json", + "payload": "", + "signatures": [{"keyid": "trust-api/rate-change-signer-2026-q4", "sig": ""}] + }, + "content_hash": "sha256:<64 hex chars>", + "signed_at": "2026-10-25T18:30:00Z" +} +``` + +- `envelope` ist der DSSE-Payload. Payload wird canonicalised (JSONC canonical bytes nach RFC 8785) und base64url (no padding) encoded; wir legen uns auf `base64url` fest. +- `content_hash` ist der SHA-256 **des gesamten DSSE-Envelopes** (also der JSON-Serialisierung mit stabilen Keys und ohne Whitespace). Dieser Hash ist unsere stabile Attestation-ID überall sonst im System. + +Fehler: +- `401` wenn kein gültiges Token. +- `403` wenn OPA die Signatur mit diesem `key_ref` nicht erlaubt (`error.code = "signing_denied"`, `error.message` = OPA-deny_reason). +- `400` wenn das Statement kein valides in-toto-v1 ist. +- `503` wenn OpenBao Transit nicht erreichbar. + +### Transparency Log — `POST /api/v1/log/entries` + +``` +POST https://trust.basicstack.de/api/v1/log/entries +Authorization: Bearer +Content-Type: application/json +``` + +Request: +```json +{ + "envelope": { ...DSSE-Envelope aus dem Signing Service... }, + "content_hash": "sha256:<64 hex chars>" +} +``` + +- `content_hash` muss mit dem SHA-256 des serialisierten `envelope` übereinstimmen — der Transparency Log rechnet nach und weist Mismatches ab. + +Response `201` (Erstanlage) **oder** `200` (Idempotenz: content_hash war bereits enthalten): +```json +{ + "log_index": 42, + "content_hash": "sha256:<...>", + "logged_at": "2026-10-25T18:30:01Z", + "inclusion_proof": { + "tree_size": 43, + "tree_root_hash": "sha256:<...>", + "signed_tree_head_ref": "sha256:<...>", + "hashes": ["sha256:<...>", "sha256:<...>", "..."] + } +} +``` + +- `log_index` ist ein monoton wachsender Integer, pro Attestation einmalig. +- `inclusion_proof.hashes` ist der Merkle-Pfad zum signierten Tree-Head (Format kompatibel zu RFC 9162 / certificate-transparency). +- `signed_tree_head_ref` referenziert den zuletzt signierten Tree-Head (den der Transparency Log periodisch veröffentlicht). Konsumenten können gegen diesen Head verifizieren. + +Idempotenz: Der Submit MUSS idempotent sein. Zweites `POST` mit identischem `content_hash` liefert `200` + die ursprünglichen Index/Proof-Daten. + +Fehler: +- `409` wenn `content_hash` nicht zum `envelope` passt. +- `401` / `403` analog zu Signing Service. +- `503` wenn das Trillian-Backend nicht erreichbar ist. + +### Beziehung zwischen den beiden Aufrufen + +Jede attestierbare Operation sieht so aus: + +``` +1. Store.Create / Store.Update (lokal, trust-postgres oder expense-postgres) + → gibt row-id zurück, rate_change_attestation_ref := NULL + +2. attest.Signer.SignDSSE(ctx, statement) + → envelope + content_hash + +3. tlog.Submitter.Submit(ctx, envelope, content_hash) + → log_index + inclusion_proof + +4. Store.AttachAttestation(ctx, row-id, content_hash) + → rate_change_attestation_ref := content_hash +``` + +Fehlerpfad: Scheitert Schritt 2 oder 3, bleibt die Row ohne `attestation_ref`. Ein nachgelagerter Reconciliation-Worker (eigenes Deployment, später) sucht periodisch nach solchen Rows und wiederholt die Attestation. Idempotenz von `/api/v1/log/entries` macht das sicher. + +## Begründung + +- **REST + JSON** statt gRPC. Debuggbar mit curl, lesbar in Logs, einfacher mTLS-Setup. gRPC-Overhead wäre bei den erwarteten Call-Raten (einzelne Signaturen pro Workflow-Schritt, nicht Mio/s) nicht gerechtfertigt. +- **DSSE als Envelope-Format**: unverändert zu `docs/architecture/v1.1.md` §4.2. +- **content_hash als einziger stabiler Rückbezug**: steht in Signatur-Envelopes, in `rate_change_attestation_ref`, in `receipt_attestation_ref`, in `computed_from.transform_attestation_ref` — konsistente Semantik, überall nachprüfbar. +- **Idempotenz-Pflicht am Log** entkoppelt Store-Transaktionen vom Netz-Erfolg. Reconciliation ist Standard-Pattern, nicht Ausnahme. +- **Pocket-ID-Token als Auth**: konsistent zum Rest der Architektur (ADR-0002); kein Zweit-IdP für diese beiden Pfade. +- **Klare Trennung** `/api/v1/sign/*` vs. `/api/v1/log/*`: die beiden Dienste leben zwar organisatorisch im `agent-trust`-Namespace, dürfen aber zu unterschiedlichen Zeiten ausfallen (OpenBao vs. Trillian), und Konsumenten müssen das unterscheiden können. + +### Alternativen bewusst nicht gewählt + +- **gRPC-first mit Protobuf**: zu viel Build-Pipeline-Komplexität für den jetzigen Scope. +- **Signatur direkt in jedem Service** (jeder hat eigenen privaten Ed25519-Schlüssel in-process): widerspricht `docs/architecture/v1.1.md` §9.1 ("Agenten dürfen private Schlüssel nicht lesen"). Zentraler Signing Service bleibt verbindlich. +- **Direkte Trillian-API** vom Konsumenten aus: Trillian's gRPC-API ist komplex, das Mapping von DSSE-Envelope auf Trillian-LogLeaf ist Boilerplate, und wir wollen Trillian austauschen können (z.B. später gegen sigstore rekor). Unsere Wrapper-API lässt das offen. + +## Konsequenzen + +### Positive + +- Konsumenten (admin-api, Agenten) brauchen nur zwei kleine HTTP-Clients mit identischem Shape. +- Idempotenz am Log entkoppelt lokale Transaktion vom Netzwerk-Rundlauf. +- Reconciliation ist entworfen, nicht nachträglich improvisiert. + +### Negative / Trade-offs + +- Zusätzlicher Hop: Store→Sign→Log→AttachAttestation statt alles in einer Transaktion. Latency-Summe im Normalfall ~50–200 ms pro Attestation; akzeptabel für Admin-Flows und Agent-Operationen. +- Reconciliation-Job ist Pflicht-Komponente, nicht optional. Muss als eigenes Deployment mit Cron oder Queue-Trigger existieren. +- Konsumenten müssen "attestation pending"-UI-Status zeigen können, damit halbfertige Rows nicht wie Fehler wirken. + +### Folgearbeiten / Open Points + +- **`services/signing-service/`**: eigenes Deployment, implementiert `/api/v1/sign/dsse`, kapselt OpenBao Transit. Kommt als Phase-2-Hauptarbeit. +- **`services/trust-api/` oder `services/transparency-log/`**: wo `/api/v1/log/entries` wirklich implementiert wird — unentschieden, hängt an Trillian-vs-eigene-Lösung-Entscheidung. Ich tendiere zu einem eigenen `transparency-log/` Service, der Trillian wrappt. +- **`services/attest-reconciler/`**: periodischer Worker, der dirty rows nachträglich attestiert. Später. +- **OPA-Policy `policies/signing/*.rego`**: welcher `client_id` darf welche `key_ref` nutzen. Später. + +## Links + +- `services/admin-api/internal/attest/` — Signing-Client (Erstkonsument, kommt in diesem Heartbeat) +- `services/admin-api/internal/tlog/` — Transparency-Log-Client (dto.) +- `docs/architecture/v1.1.md` §4.2, §4.5a, §8.3 +- RFC 9162 (Certificate Transparency v2) +- DSSE Spec: https://github.com/secure-systems-lab/dsse +- in-toto Attestation Framework v1.0: https://in-toto.io diff --git a/docs/adr/README.md b/docs/adr/README.md index ab6f065..5cb3a45 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -17,5 +17,6 @@ Dieses Verzeichnis dokumentiert durchlebte Architekturentscheidungen in [MADR-Fo | 0002 | [Session-Scope-Token-Format](0002-session-scope-token-format.md) | accepted | | 0003 | [PII-Retention-Richtlinie](0003-pii-retention-policy.md) | accepted | | 0004 | [Datenbank-Migrationsframework (golang-migrate)](0004-db-migrations.md) | accepted | +| 0005 | [Signing-Service- und Transparency-Log-API-Vertrag](0005-signing-service-and-transparency-log-api.md) | accepted | Vorlage: [`0000-template.md`](0000-template.md). diff --git a/services/admin-api/internal/attest/hash.go b/services/admin-api/internal/attest/hash.go new file mode 100644 index 0000000..121de36 --- /dev/null +++ b/services/admin-api/internal/attest/hash.go @@ -0,0 +1,93 @@ +package attest + +import ( + "bytes" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "fmt" + "sort" +) + +// sha256hex returns the hex-encoded SHA-256 of b. +func sha256hex(b []byte) string { + sum := sha256.Sum256(b) + return hex.EncodeToString(sum[:]) +} + +// jsonMarshalSortedKeys produces a stable JSON encoding with keys sorted +// recursively in every object. This is the stability we need for hashing; +// it is NOT a full RFC 8785 implementation (no number-canonicalisation, +// no string-escape canonicalisation), but all values we hash are strings, +// nulls, maps, or arrays of those, which stay stable under encoding/json. +func jsonMarshalSortedKeys(v any) ([]byte, error) { + // Round-trip through generic unmarshal to get the canonical shape, then + // emit sorted. For our (small, well-shaped) inputs this is cheap. + first, err := json.Marshal(v) + if err != nil { + return nil, err + } + var any_ any + if err := json.Unmarshal(first, &any_); err != nil { + return nil, err + } + var buf bytes.Buffer + if err := writeSorted(&buf, any_); err != nil { + return nil, err + } + return buf.Bytes(), nil +} + +func writeSorted(buf *bytes.Buffer, v any) error { + switch x := v.(type) { + case nil: + buf.WriteString("null") + case bool: + if x { + buf.WriteString("true") + } else { + buf.WriteString("false") + } + case float64: + // json.Unmarshal gives floats for all numbers. In our payloads we + // never emit raw numbers; everything is a string. + b, _ := json.Marshal(x) + buf.Write(b) + case string: + b, _ := json.Marshal(x) + buf.Write(b) + case []any: + buf.WriteByte('[') + for i, el := range x { + if i > 0 { + buf.WriteByte(',') + } + if err := writeSorted(buf, el); err != nil { + return err + } + } + buf.WriteByte(']') + case map[string]any: + keys := make([]string, 0, len(x)) + for k := range x { + keys = append(keys, k) + } + sort.Strings(keys) + buf.WriteByte('{') + for i, k := range keys { + if i > 0 { + buf.WriteByte(',') + } + kb, _ := json.Marshal(k) + buf.Write(kb) + buf.WriteByte(':') + if err := writeSorted(buf, x[k]); err != nil { + return err + } + } + buf.WriteByte('}') + default: + return fmt.Errorf("writeSorted: unsupported type %T", v) + } + return nil +} diff --git a/services/admin-api/internal/attest/http_signer.go b/services/admin-api/internal/attest/http_signer.go new file mode 100644 index 0000000..c2fab82 --- /dev/null +++ b/services/admin-api/internal/attest/http_signer.go @@ -0,0 +1,85 @@ +package attest + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "fmt" + "io" + "net/http" + "time" +) + +// HTTPSigner calls POST /api/v1/sign/dsse on the Signing Service per ADR-0005. +// Transport is reused across calls; configure via Client. +type HTTPSigner struct { + BaseURL string // e.g. https://trust.basicstack.de + Client *http.Client + TokenSource TokenSource // supplies Bearer tokens for the Pocket ID client_credentials grant +} + +// TokenSource returns a valid Pocket ID access token. Implementations are +// expected to cache + refresh; HTTPSigner calls Token on every request. +type TokenSource interface { + Token(ctx context.Context) (string, error) +} + +// StaticToken is a trivial TokenSource for tests and development. +type StaticToken string + +func (s StaticToken) Token(_ context.Context) (string, error) { return string(s), nil } + +type signRequest struct { + KeyRef string `json:"key_ref"` + Statement Statement `json:"statement"` +} + +type signResponse struct { + Envelope Envelope `json:"envelope"` + ContentHash string `json:"content_hash"` + SignedAt time.Time `json:"signed_at"` +} + +func (s HTTPSigner) SignDSSE(ctx context.Context, keyRef string, stmt Statement) (Envelope, string, error) { + if s.BaseURL == "" { + return Envelope{}, "", errors.New("attest.HTTPSigner: BaseURL is empty") + } + cli := s.Client + if cli == nil { + cli = &http.Client{Timeout: 10 * time.Second} + } + body, err := json.Marshal(signRequest{KeyRef: keyRef, Statement: stmt}) + if err != nil { + return Envelope{}, "", fmt.Errorf("marshal: %w", err) + } + req, err := http.NewRequestWithContext(ctx, http.MethodPost, s.BaseURL+"/api/v1/sign/dsse", bytes.NewReader(body)) + if err != nil { + return Envelope{}, "", err + } + req.Header.Set("Content-Type", "application/json") + if s.TokenSource != nil { + tok, err := s.TokenSource.Token(ctx) + if err != nil { + return Envelope{}, "", fmt.Errorf("token: %w", err) + } + req.Header.Set("Authorization", "Bearer "+tok) + } + resp, err := cli.Do(req) + if err != nil { + return Envelope{}, "", err + } + defer resp.Body.Close() + raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20)) + if resp.StatusCode != http.StatusOK { + return Envelope{}, "", fmt.Errorf("signer: status %d: %s", resp.StatusCode, string(raw)) + } + var out signResponse + if err := json.Unmarshal(raw, &out); err != nil { + return Envelope{}, "", fmt.Errorf("decode: %w", err) + } + if out.ContentHash == "" { + return Envelope{}, "", errors.New("signer: response missing content_hash") + } + return out.Envelope, out.ContentHash, nil +} diff --git a/services/admin-api/internal/attest/http_signer_test.go b/services/admin-api/internal/attest/http_signer_test.go new file mode 100644 index 0000000..06fb7fe --- /dev/null +++ b/services/admin-api/internal/attest/http_signer_test.go @@ -0,0 +1,126 @@ +package attest + +import ( + "context" + "encoding/json" + "net/http" + "net/http/httptest" + "strings" + "testing" + "time" + + "basicstack.de/proofedmesh/services/admin-api/internal/model" +) + +func sampleRate(t *testing.T) model.Rate { + t.Helper() + validFrom, _ := time.Parse("2006-01-02", "2026-01-01") + return model.Rate{ + RateID: "PDR-1", Country: "DE", Currency: "EUR", + FullDay: "28.00", PartialDayMin8h: "14.00", + ArrivalDay: "14.00", DepartureDay: "14.00", + ValidFrom: validFrom, + SourceRegulation: "BMF test", + CreatedBy: "admin", + } +} + +func TestHTTPSigner_HappyPath(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/api/v1/sign/dsse" { + t.Errorf("unexpected path %s", r.URL.Path) + } + if auth := r.Header.Get("Authorization"); auth != "Bearer test-token" { + t.Errorf("auth header: %q", auth) + } + var req signRequest + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + t.Fatalf("decode: %v", err) + } + if req.KeyRef != "trust-api/test-key" { + t.Errorf("key_ref: %q", req.KeyRef) + } + if req.Statement.PredicateType != "https://basicstack.de/attestations/rate_change/v1" { + t.Errorf("predicate type: %q", req.Statement.PredicateType) + } + // Fake envelope + content_hash + env := Envelope{ + PayloadType: "application/vnd.in-toto+json", + Payload: "ZmFrZS1wYXlsb2Fk", // "fake-payload" + Signatures: []EnvelopeSignature{{KeyID: req.KeyRef, Sig: "ZmFrZS1zaWc"}}, + } + ch, _ := EnvelopeContentHash(env) + resp := signResponse{ + Envelope: env, + ContentHash: ch, + SignedAt: time.Now().UTC(), + } + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(resp) + })) + defer srv.Close() + + s := HTTPSigner{BaseURL: srv.URL, TokenSource: StaticToken("test-token")} + stmt, _ := buildStatement(RateChangeEvent{ + Kind: ChangeCreate, + Rate: sampleRate(t), + PerformedBy: PerformedBy{Issuer: "https://auth.basicstack.de", Subject: "admin-sub"}, + PerformedAt: time.Now().UTC(), + }) + env, ch, err := s.SignDSSE(context.Background(), "trust-api/test-key", stmt) + if err != nil { + t.Fatalf("sign: %v", err) + } + if !strings.HasPrefix(ch, "sha256:") || len(ch) != 71 { + t.Errorf("content_hash shape: %q", ch) + } + if env.PayloadType != "application/vnd.in-toto+json" { + t.Errorf("payload_type: %q", env.PayloadType) + } +} + +func TestHTTPSigner_SurfacesServerError(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Error(w, `{"error":{"code":"signing_denied","message":"OPA deny: no such key"}}`, http.StatusForbidden) + })) + defer srv.Close() + s := HTTPSigner{BaseURL: srv.URL, TokenSource: StaticToken("t")} + stmt, _ := buildStatement(RateChangeEvent{Kind: ChangeCreate, Rate: sampleRate(t)}) + _, _, err := s.SignDSSE(context.Background(), "forbidden-key", stmt) + if err == nil { + t.Fatal("expected error from 403") + } + if !strings.Contains(err.Error(), "signing_denied") { + t.Errorf("expected server message in error, got: %v", err) + } +} + +func TestHTTPSigner_RejectsEmptyBaseURL(t *testing.T) { + s := HTTPSigner{} + _, _, err := s.SignDSSE(context.Background(), "k", Statement{}) + if err == nil { + t.Fatal("expected error for empty BaseURL") + } +} + +func TestBuildStatement_StableRateHash(t *testing.T) { + r := sampleRate(t) + ev := RateChangeEvent{Kind: ChangeCreate, Rate: r, PerformedAt: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC)} + a, err := buildStatement(ev) + if err != nil { + t.Fatal(err) + } + b, err := buildStatement(ev) + if err != nil { + t.Fatal(err) + } + if a.Subject[0].Digest["sha256"] != b.Subject[0].Digest["sha256"] { + t.Fatalf("rate hash is not stable:\n a=%s\n b=%s", a.Subject[0].Digest["sha256"], b.Subject[0].Digest["sha256"]) + } + // Changing the amount changes the hash. + r.FullDay = "99.00" + c, _ := buildStatement(RateChangeEvent{Kind: ChangeCreate, Rate: r, PerformedAt: ev.PerformedAt}) + if a.Subject[0].Digest["sha256"] == c.Subject[0].Digest["sha256"] { + t.Fatal("rate hash collided on amount change") + } +} diff --git a/services/admin-api/internal/attest/orchestrator_test.go b/services/admin-api/internal/attest/orchestrator_test.go new file mode 100644 index 0000000..2609de9 --- /dev/null +++ b/services/admin-api/internal/attest/orchestrator_test.go @@ -0,0 +1,131 @@ +package attest + +import ( + "context" + "errors" + "testing" + "time" + + "basicstack.de/proofedmesh/services/admin-api/internal/model" +) + +type fakeSigner struct { + err error + wantKeyRef string + tookStmt Statement + returnEnv Envelope + returnHash string +} + +func (f *fakeSigner) SignDSSE(_ context.Context, keyRef string, stmt Statement) (Envelope, string, error) { + if f.err != nil { + return Envelope{}, "", f.err + } + f.tookStmt = stmt + if f.wantKeyRef != "" && keyRef != f.wantKeyRef { + return Envelope{}, "", errors.New("unexpected keyRef") + } + return f.returnEnv, f.returnHash, nil +} + +type fakeSubmitter struct { + err error + gotEnv Envelope + gotHash string + returnRct LogReceipt +} + +func (f *fakeSubmitter) Submit(_ context.Context, env Envelope, ch string) (LogReceipt, error) { + if f.err != nil { + return LogReceipt{}, f.err + } + f.gotEnv = env + f.gotHash = ch + if f.returnRct.ContentHash == "" { + f.returnRct.ContentHash = ch + } + return f.returnRct, nil +} + +func TestOrchestrator_HappyPath(t *testing.T) { + env := Envelope{PayloadType: "application/vnd.in-toto+json", Payload: "aGVsbG8"} + ch, _ := EnvelopeContentHash(env) + s := &fakeSigner{wantKeyRef: "k", returnEnv: env, returnHash: ch} + sub := &fakeSubmitter{returnRct: LogReceipt{LogIndex: 7, ContentHash: ch, LoggedAt: time.Now()}} + orch := Orchestrator{Signer: s, Submitter: sub, KeyRef: "k"} + + gotHash, receipt, err := orch.SignAndLog(context.Background(), RateChangeEvent{ + Kind: ChangeCreate, + Rate: sampleRateMinimal(), + PerformedBy: PerformedBy{Issuer: "https://auth.basicstack.de", Subject: "u", SessionID: "s"}, + PerformedAt: time.Now(), + }) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if gotHash != ch { + t.Errorf("hash mismatch: %q vs %q", gotHash, ch) + } + if receipt.LogIndex != 7 { + t.Errorf("receipt log_index: %d", receipt.LogIndex) + } + if sub.gotHash != ch { + t.Errorf("submitter saw different hash: %q vs %q", sub.gotHash, ch) + } + if s.tookStmt.PredicateType != "https://basicstack.de/attestations/rate_change/v1" { + t.Errorf("predicateType in statement: %q", s.tookStmt.PredicateType) + } +} + +func TestOrchestrator_SignerErrorStops(t *testing.T) { + orch := Orchestrator{ + Signer: &fakeSigner{err: errors.New("bao down")}, + Submitter: &fakeSubmitter{}, + KeyRef: "k", + } + _, _, err := orch.SignAndLog(context.Background(), RateChangeEvent{Rate: sampleRateMinimal()}) + if err == nil { + t.Fatal("expected error") + } +} + +func TestOrchestrator_SubmitterErrorPropagates(t *testing.T) { + env := Envelope{PayloadType: "application/vnd.in-toto+json"} + ch, _ := EnvelopeContentHash(env) + orch := Orchestrator{ + Signer: &fakeSigner{wantKeyRef: "k", returnEnv: env, returnHash: ch}, + Submitter: &fakeSubmitter{err: errors.New("trillian down")}, + KeyRef: "k", + } + gotHash, _, err := orch.SignAndLog(context.Background(), RateChangeEvent{Rate: sampleRateMinimal()}) + if err == nil { + t.Fatal("expected error") + } + // The hash from the signer is still returned so a reconciler can retry. + if gotHash != ch { + t.Errorf("expected signer hash to leak through on submitter error, got %q", gotHash) + } +} + +func TestOrchestrator_RejectsMisconfiguration(t *testing.T) { + cases := []Orchestrator{ + {Signer: nil, Submitter: &fakeSubmitter{}, KeyRef: "k"}, + {Signer: &fakeSigner{}, Submitter: nil, KeyRef: "k"}, + {Signer: &fakeSigner{}, Submitter: &fakeSubmitter{}, KeyRef: ""}, + } + for i, o := range cases { + if _, _, err := o.SignAndLog(context.Background(), RateChangeEvent{}); err == nil { + t.Errorf("case %d: expected error for misconfigured orchestrator", i) + } + } +} + +func sampleRateMinimal() model.Rate { + t, _ := time.Parse("2006-01-02", "2026-01-01") + return model.Rate{ + RateID: "PDR-1", Country: "DE", Currency: "EUR", + FullDay: "28.00", PartialDayMin8h: "14.00", + ArrivalDay: "14.00", DepartureDay: "14.00", + ValidFrom: t, SourceRegulation: "test", CreatedBy: "admin", + } +} diff --git a/services/admin-api/internal/attest/ratechange.go b/services/admin-api/internal/attest/ratechange.go index aa17e0a..c9cde9f 100644 --- a/services/admin-api/internal/attest/ratechange.go +++ b/services/admin-api/internal/attest/ratechange.go @@ -1,15 +1,78 @@ -// Package attest hands off a rate-change event to the Signing Service and -// returns the SHA-256 of the signed attestation envelope. Current state: -// Stub — PERSUB will be wired in Phase-2. +// Package attest signs rate_change events and submits the resulting DSSE +// envelope to the Transparency Log. Current state: HTTP clients fully wired +// against the API contract from ADR-0005. Both services (signing-service, +// transparency-log) are still to be deployed; the clients are already usable +// against httptest servers and will plug into real services unchanged. package attest import ( "context" + "encoding/base64" + "encoding/json" "errors" + "fmt" + "time" "basicstack.de/proofedmesh/services/admin-api/internal/model" ) +// Statement is the in-toto v1 Statement we hand to the Signing Service. +// We serialise it to the Signing Service as the request's `statement` field. +type Statement struct { + Type string `json:"_type"` + Subject []Subject `json:"subject"` + PredicateType string `json:"predicateType"` + Predicate map[string]any `json:"predicate"` +} + +type Subject struct { + Name string `json:"name"` + Digest map[string]string `json:"digest"` +} + +// Envelope is the DSSE envelope returned by the Signing Service. +type Envelope struct { + PayloadType string `json:"payloadType"` + Payload string `json:"payload"` // base64url + Signatures []EnvelopeSignature `json:"signatures"` +} + +type EnvelopeSignature struct { + KeyID string `json:"keyid"` + Sig string `json:"sig"` +} + +// Signer wraps the "POST /api/v1/sign/dsse" call. Returned content_hash is the +// SHA-256 of the serialised envelope; it IS the attestation reference used +// everywhere else in the system. +type Signer interface { + SignDSSE(ctx context.Context, keyRef string, stmt Statement) (envelope Envelope, contentHash string, err error) +} + +// Submitter wraps the "POST /api/v1/log/entries" call on the Transparency Log. +// MUST be idempotent: calling twice with the same (envelope, content_hash) +// returns the same log_index and the same inclusion proof. +type Submitter interface { + Submit(ctx context.Context, env Envelope, contentHash string) (LogReceipt, error) +} + +// LogReceipt is what the Transparency Log hands us back. +type LogReceipt struct { + LogIndex int64 `json:"log_index"` + ContentHash string `json:"content_hash"` + LoggedAt time.Time `json:"logged_at"` + InclusionProof InclusionProof `json:"inclusion_proof"` +} + +type InclusionProof struct { + TreeSize int64 `json:"tree_size"` + TreeRootHash string `json:"tree_root_hash"` + SignedTreeHeadRef string `json:"signed_tree_head_ref"` + Hashes []string `json:"hashes"` +} + +// ChangeKind labels why a rate changed. Mirrored into the predicate so that +// the Verification Service can reason about it later. type ChangeKind string const ( @@ -17,33 +80,158 @@ const ( ChangeRetire ChangeKind = "retire" ) -// RateChangeEvent is what we hand to the Signing Service. +// RateChangeEvent is what the admin-api hands to SignAndLog on every mutation. type RateChangeEvent struct { - Kind ChangeKind - Rate model.Rate - PerformedBy PerformedBy - PerformedAtRFC3339 string + Kind ChangeKind + Rate model.Rate + PerformedBy PerformedBy + PerformedAt time.Time } type PerformedBy struct { - Issuer string - Subject string - SessionID string + Issuer string `json:"iss"` + Subject string `json:"sub"` + SessionID string `json:"session_id"` } -// Signer issues a signed rate_change attestation and returns the content hash -// (sha256:...) under which it is stored in the Transparency Log. -type Signer interface { - SignRateChange(ctx context.Context, ev RateChangeEvent) (attestationRef string, err error) +// SignAndLog orchestrates the Signing Service call and the Transparency Log +// submit. Returns the attestation reference (sha256:) on success. If +// signing succeeds but logging fails, SignAndLog returns the error AND the +// (partial) envelope so a reconciliation worker can retry just the submit. +type Orchestrator struct { + Signer Signer + Submitter Submitter + KeyRef string // the signing key reference, e.g. "trust-api/rate-change-signer-2026-q4" } -// NopSigner is a placeholder that lets the admin-api start without a real -// Signing Service. It must not be used in production — the HTTP layer gates -// its usage behind a startup flag. -type NopSigner struct{} +func (o Orchestrator) SignAndLog(ctx context.Context, ev RateChangeEvent) (string, LogReceipt, error) { + if o.Signer == nil { + return "", LogReceipt{}, errors.New("attest.Orchestrator: Signer is nil") + } + if o.Submitter == nil { + return "", LogReceipt{}, errors.New("attest.Orchestrator: Submitter is nil") + } + if o.KeyRef == "" { + return "", LogReceipt{}, errors.New("attest.Orchestrator: KeyRef is empty") + } -var errNopNotForProd = errors.New("attest.NopSigner is a stub and must not be used in production") - -func (NopSigner) SignRateChange(ctx context.Context, ev RateChangeEvent) (string, error) { - return "", errNopNotForProd + stmt, err := buildStatement(ev) + if err != nil { + return "", LogReceipt{}, fmt.Errorf("build statement: %w", err) + } + env, contentHash, err := o.Signer.SignDSSE(ctx, o.KeyRef, stmt) + if err != nil { + return "", LogReceipt{}, fmt.Errorf("sign dsse: %w", err) + } + receipt, err := o.Submitter.Submit(ctx, env, contentHash) + if err != nil { + return contentHash, LogReceipt{}, fmt.Errorf("submit tlog: %w", err) + } + if receipt.ContentHash != contentHash { + return contentHash, receipt, fmt.Errorf("tlog returned content_hash %q, expected %q", receipt.ContentHash, contentHash) + } + return contentHash, receipt, nil } + +// buildStatement produces a canonical in-toto v1 Statement for a rate change. +// The subject.name encodes rate_id for immediate human readability; the +// subject.digest is the SHA-256 of the rate's content-defining fields so that +// verifiers can rehash the stored row and confirm the signature covers it. +func buildStatement(ev RateChangeEvent) (Statement, error) { + rateHash, err := hashRateCore(ev.Rate) + if err != nil { + return Statement{}, err + } + predicate := map[string]any{ + "change_kind": string(ev.Kind), + "rate_id": ev.Rate.RateID, + "country": ev.Rate.Country, + "city": ev.Rate.City, + "currency": ev.Rate.Currency, + "full_day": ev.Rate.FullDay, + "partial_day_min8h": ev.Rate.PartialDayMin8h, + "arrival_day": ev.Rate.ArrivalDay, + "departure_day": ev.Rate.DepartureDay, + "valid_from": ev.Rate.ValidFrom.Format("2006-01-02"), + "valid_to": nilDate(ev.Rate.ValidTo), + "source_regulation": ev.Rate.SourceRegulation, + "performed_by": ev.PerformedBy, + "performed_at": ev.PerformedAt.UTC().Format(time.RFC3339), + } + return Statement{ + Type: "https://in-toto.io/Statement/v1", + Subject: []Subject{{ + Name: fmt.Sprintf("per_diem_rate/%s", ev.Rate.RateID), + Digest: map[string]string{"sha256": rateHash}, + }}, + PredicateType: "https://basicstack.de/attestations/rate_change/v1", + Predicate: predicate, + }, nil +} + +func nilDate(t *time.Time) any { + if t == nil { + return nil + } + return t.Format("2006-01-02") +} + +// hashRateCore computes a stable SHA-256 over the fields that identify a rate. +// The exclusion of valid_to and the attestation ref itself is deliberate: both +// may legitimately change later (retire + attach_attestation) and must not +// invalidate the historical signature. +func hashRateCore(r model.Rate) (string, error) { + core := map[string]any{ + "rate_id": r.RateID, + "country": r.Country, + "city": r.City, + "currency": r.Currency, + "full_day": r.FullDay, + "partial_day_min8h": r.PartialDayMin8h, + "arrival_day": r.ArrivalDay, + "departure_day": r.DepartureDay, + "valid_from": r.ValidFrom.Format("2006-01-02"), + "source_regulation": r.SourceRegulation, + } + b, err := canonicalJSON(core) + if err != nil { + return "", err + } + return "sha256:" + sha256hex(b), nil +} + +// canonicalJSON writes v with sorted object keys, no insignificant whitespace, +// matching the subset of RFC 8785 we need for stable hashing. +func canonicalJSON(v any) ([]byte, error) { + return jsonMarshalSortedKeys(v) +} + +// base64urlStd is an alias so tests can refer to the same encoding the real +// signing service uses on the wire. +var base64urlStd = base64.RawURLEncoding + +// EncodePayload returns the base64url-no-padding encoding of the canonical +// statement bytes. Exposed for in-process test signers. +func EncodePayload(stmt Statement) (string, []byte, error) { + b, err := canonicalJSON(stmt) + if err != nil { + return "", nil, err + } + return base64urlStd.EncodeToString(b), b, nil +} + +// EnvelopeContentHash returns the SHA-256 of the canonical JSON of env. +// Used by both the Transparency Log server (to verify what the client sent) +// and by in-process test signers (to produce a correct content_hash). +func EnvelopeContentHash(env Envelope) (string, error) { + b, err := canonicalJSON(env) + if err != nil { + return "", err + } + return "sha256:" + sha256hex(b), nil +} + +// --- helpers moved to a separate file (hash.go) to keep this one readable --- + +// Compile-time assertion that the serialisation layer actually gets wired. +var _ = json.Marshal diff --git a/services/admin-api/internal/tlog/submitter.go b/services/admin-api/internal/tlog/submitter.go new file mode 100644 index 0000000..e35e597 --- /dev/null +++ b/services/admin-api/internal/tlog/submitter.go @@ -0,0 +1,80 @@ +// Package tlog contains the client for the Transparency Log HTTP API per +// ADR-0005. The server-side implementation (Trillian wrapper) lives in its +// own service and is not part of admin-api. +package tlog + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "fmt" + "io" + "net/http" + "time" + + "basicstack.de/proofedmesh/services/admin-api/internal/attest" +) + +// HTTPSubmitter calls POST /api/v1/log/entries on the Transparency Log. +// Idempotent on the server side: resubmitting the same (envelope, content_hash) +// is safe and returns the same log_index + inclusion_proof. +type HTTPSubmitter struct { + BaseURL string + Client *http.Client + TokenSource attest.TokenSource +} + +type submitRequest struct { + Envelope attest.Envelope `json:"envelope"` + ContentHash string `json:"content_hash"` +} + +func (s HTTPSubmitter) Submit(ctx context.Context, env attest.Envelope, contentHash string) (attest.LogReceipt, error) { + if s.BaseURL == "" { + return attest.LogReceipt{}, errors.New("tlog.HTTPSubmitter: BaseURL is empty") + } + cli := s.Client + if cli == nil { + cli = &http.Client{Timeout: 10 * time.Second} + } + body, err := json.Marshal(submitRequest{Envelope: env, ContentHash: contentHash}) + if err != nil { + return attest.LogReceipt{}, fmt.Errorf("marshal: %w", err) + } + req, err := http.NewRequestWithContext(ctx, http.MethodPost, s.BaseURL+"/api/v1/log/entries", bytes.NewReader(body)) + if err != nil { + return attest.LogReceipt{}, err + } + req.Header.Set("Content-Type", "application/json") + if s.TokenSource != nil { + tok, err := s.TokenSource.Token(ctx) + if err != nil { + return attest.LogReceipt{}, fmt.Errorf("token: %w", err) + } + req.Header.Set("Authorization", "Bearer "+tok) + } + resp, err := cli.Do(req) + if err != nil { + return attest.LogReceipt{}, err + } + defer resp.Body.Close() + raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20)) + switch resp.StatusCode { + case http.StatusOK, http.StatusCreated: + // ok (201 first-time, 200 idempotent repeat) + default: + return attest.LogReceipt{}, fmt.Errorf("tlog: status %d: %s", resp.StatusCode, string(raw)) + } + var out attest.LogReceipt + if err := json.Unmarshal(raw, &out); err != nil { + return attest.LogReceipt{}, fmt.Errorf("decode: %w", err) + } + if out.ContentHash != contentHash { + return out, fmt.Errorf("tlog response content_hash %q != sent %q", out.ContentHash, contentHash) + } + if out.LogIndex < 0 { + return out, fmt.Errorf("tlog response has negative log_index %d", out.LogIndex) + } + return out, nil +} diff --git a/services/admin-api/internal/tlog/submitter_test.go b/services/admin-api/internal/tlog/submitter_test.go new file mode 100644 index 0000000..01b7950 --- /dev/null +++ b/services/admin-api/internal/tlog/submitter_test.go @@ -0,0 +1,137 @@ +package tlog + +import ( + "context" + "encoding/json" + "net/http" + "net/http/httptest" + "strings" + "sync" + "testing" + "time" + + "basicstack.de/proofedmesh/services/admin-api/internal/attest" +) + +type fakeTLogServer struct { + mu sync.Mutex + byHash map[string]attest.LogReceipt + nextIdx int64 +} + +func newFakeServer() *fakeTLogServer { + return &fakeTLogServer{byHash: make(map[string]attest.LogReceipt)} +} + +func (f *fakeTLogServer) handler(t *testing.T) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/api/v1/log/entries" { + t.Errorf("unexpected path %s", r.URL.Path) + } + var req submitRequest + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + http.Error(w, err.Error(), http.StatusBadRequest) + return + } + // Verify content_hash matches the envelope on the wire. + recomputed, _ := attest.EnvelopeContentHash(req.Envelope) + if recomputed != req.ContentHash { + http.Error(w, `{"error":"content_hash mismatch"}`, http.StatusConflict) + return + } + f.mu.Lock() + defer f.mu.Unlock() + if existing, ok := f.byHash[req.ContentHash]; ok { + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusOK) // idempotent repeat + _ = json.NewEncoder(w).Encode(existing) + return + } + f.nextIdx++ + receipt := attest.LogReceipt{ + LogIndex: f.nextIdx, + ContentHash: req.ContentHash, + LoggedAt: time.Now().UTC(), + InclusionProof: attest.InclusionProof{ + TreeSize: f.nextIdx + 1, + TreeRootHash: "sha256:" + strings.Repeat("a", 64), + SignedTreeHeadRef: "sha256:" + strings.Repeat("b", 64), + Hashes: []string{"sha256:" + strings.Repeat("c", 64)}, + }, + } + f.byHash[req.ContentHash] = receipt + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusCreated) + _ = json.NewEncoder(w).Encode(receipt) + }) +} + +func sampleEnvelope() attest.Envelope { + return attest.Envelope{ + PayloadType: "application/vnd.in-toto+json", + Payload: "aGVsbG8", // "hello" (base64url) + Signatures: []attest.EnvelopeSignature{{KeyID: "k", Sig: "c2ln"}}, + } +} + +func TestSubmit_HappyPathAndIdempotency(t *testing.T) { + f := newFakeServer() + srv := httptest.NewServer(f.handler(t)) + defer srv.Close() + s := HTTPSubmitter{BaseURL: srv.URL, TokenSource: attest.StaticToken("t")} + env := sampleEnvelope() + ch, err := attest.EnvelopeContentHash(env) + if err != nil { + t.Fatalf("content_hash: %v", err) + } + r1, err := s.Submit(context.Background(), env, ch) + if err != nil { + t.Fatalf("first submit: %v", err) + } + if r1.LogIndex != 1 { + t.Errorf("first log_index: %d", r1.LogIndex) + } + // Re-submit must return the SAME log_index. + r2, err := s.Submit(context.Background(), env, ch) + if err != nil { + t.Fatalf("second submit: %v", err) + } + if r2.LogIndex != 1 { + t.Errorf("idempotent repeat log_index changed: %d", r2.LogIndex) + } +} + +func TestSubmit_RejectsContentHashMismatch(t *testing.T) { + f := newFakeServer() + srv := httptest.NewServer(f.handler(t)) + defer srv.Close() + s := HTTPSubmitter{BaseURL: srv.URL, TokenSource: attest.StaticToken("t")} + env := sampleEnvelope() + _, err := s.Submit(context.Background(), env, "sha256:"+strings.Repeat("f", 64)) + if err == nil { + t.Fatal("expected server to reject content_hash mismatch") + } +} + +func TestSubmit_SurfacesServerErrors(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Error(w, "trillian down", http.StatusServiceUnavailable) + })) + defer srv.Close() + s := HTTPSubmitter{BaseURL: srv.URL, TokenSource: attest.StaticToken("t")} + _, err := s.Submit(context.Background(), sampleEnvelope(), "sha256:"+strings.Repeat("a", 64)) + if err == nil { + t.Fatal("expected error from 503") + } + if !strings.Contains(err.Error(), "status 503") { + t.Errorf("expected status in error, got %v", err) + } +} + +func TestSubmit_RejectsEmptyBaseURL(t *testing.T) { + s := HTTPSubmitter{} + _, err := s.Submit(context.Background(), sampleEnvelope(), "sha256:"+strings.Repeat("a", 64)) + if err == nil { + t.Fatal("expected error for empty BaseURL") + } +}