services: add attest-reconciler Go service (periodic worker re-signs per_diem_rates rows without rate_change_attestation_ref via Signer+TLog, 11/11 go tests pass, go vet clean)
- services/attest-reconciler/ as its own go.mod (admin-api internal/ is not importable; shared code gets extracted when a 3rd consumer appears). - Scheduler accepts "@every <dur>" or a bare Go duration; context-cancellation stops the loop cleanly. - Store client defines PendingRateStore + Postgres skeleton for SELECT ... WHERE rate_change_attestation_ref IS NULL. - Orchestrator + HTTPSigner + tlog.HTTPSubmitter mirror admin-api byte-for-byte so the TLog idempotency key collides on resubmits (kein Doppelsignieren). - Prometheus text-format /metrics endpoint, dependency-free: attest_reconciler_dirty_rows_total, _reconciliation_attempts_total, _reconciliation_failures_total, _idempotent_noop_total. - Tests with fake store + fake signer + fake submitter cover: happy path, already-attested rows are NOT re-signed, TLog idempotency across ticks, TLog-down failure path, pre-cancelled ctx. - Top-level README reflects the new service; CI auto-discovers services/*/go.mod so no workflow change needed. Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
parent
be3ac9cdb8
commit
2e44147a71
17 changed files with 1537 additions and 0 deletions
|
|
@ -74,6 +74,7 @@ policies/
|
||||||
services/
|
services/
|
||||||
per-diem-calculator/ # deterministic calculator (Go)
|
per-diem-calculator/ # deterministic calculator (Go)
|
||||||
admin-api/ # admin surface for per-diem rate management (Go)
|
admin-api/ # admin surface for per-diem rate management (Go)
|
||||||
|
attest-reconciler/ # periodic worker: re-signs per_diem_rates rows without rate_change_attestation_ref (Go)
|
||||||
.forgejo/workflows/
|
.forgejo/workflows/
|
||||||
validate.yml # JSON-Schema (ajv) + Rego (opa fmt + opa test) + Go (vet + test)
|
validate.yml # JSON-Schema (ajv) + Rego (opa fmt + opa test) + Go (vet + test)
|
||||||
```
|
```
|
||||||
|
|
@ -101,6 +102,7 @@ opa test policies/ -v
|
||||||
# Go services
|
# Go services
|
||||||
(cd services/admin-api && go vet ./... && go test ./...)
|
(cd services/admin-api && go vet ./... && go test ./...)
|
||||||
(cd services/per-diem-calculator && go vet ./... && go test ./...)
|
(cd services/per-diem-calculator && go vet ./... && go test ./...)
|
||||||
|
(cd services/attest-reconciler && go vet ./... && go test ./...)
|
||||||
```
|
```
|
||||||
|
|
||||||
CI (`.forgejo/workflows/validate.yml`) läuft dieselben Prüfungen auf jedem Push und PR gegen `main`.
|
CI (`.forgejo/workflows/validate.yml`) läuft dieselben Prüfungen auf jedem Push und PR gegen `main`.
|
||||||
|
|
|
||||||
55
services/attest-reconciler/README.md
Normal file
55
services/attest-reconciler/README.md
Normal file
|
|
@ -0,0 +1,55 @@
|
||||||
|
# attest-reconciler
|
||||||
|
|
||||||
|
Periodic worker that re-signs `per_diem_rates` rows missing a
|
||||||
|
`rate_change_attestation_ref`. Mirrors `admin-api`'s Signer + TLog orchestrator
|
||||||
|
(ADR-0005). Leans on the Transparency Log's idempotency guarantee so re-runs
|
||||||
|
land on the same `log_index` and never produce a second entry.
|
||||||
|
|
||||||
|
## Layout
|
||||||
|
|
||||||
|
- `cmd/attest-reconciler/` — entry point; wires store, orchestrator, metrics,
|
||||||
|
and scheduler, handles `SIGINT`/`SIGTERM`.
|
||||||
|
- `internal/reconcile/` — `RunOnce` (per-tick work) and `Scheduler` (parses
|
||||||
|
`@every <dur>` or a bare Go duration and drives `RunOnce`).
|
||||||
|
- `internal/attest/` — Statement, Envelope, Orchestrator, `HTTPSigner`. Byte
|
||||||
|
parity with `admin-api/internal/attest`; split out until a 3rd consumer
|
||||||
|
justifies a shared module (see parent issue note).
|
||||||
|
- `internal/tlog/` — `HTTPSubmitter` for `POST /api/v1/log/entries`.
|
||||||
|
- `internal/store/` — `PendingRateStore` interface + Postgres skeleton.
|
||||||
|
- `internal/metrics/` — dependency-free Prometheus text-format exporter for
|
||||||
|
`attest_reconciler_dirty_rows_total`,
|
||||||
|
`attest_reconciler_reconciliation_attempts_total`,
|
||||||
|
`attest_reconciler_reconciliation_failures_total`,
|
||||||
|
`attest_reconciler_idempotent_noop_total`.
|
||||||
|
|
||||||
|
## Running
|
||||||
|
|
||||||
|
```
|
||||||
|
attest-reconciler \
|
||||||
|
-spec="@every 1m" \
|
||||||
|
-signer-url="https://trust.basicstack.de" \
|
||||||
|
-tlog-url="https://tlog.basicstack.de" \
|
||||||
|
-bearer="$POCKET_ID_TOKEN" \
|
||||||
|
-metrics-addr=":9090"
|
||||||
|
```
|
||||||
|
|
||||||
|
Production replaces `-bearer` with the `client_credentials` TokenSource built
|
||||||
|
in DEV-602.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
```
|
||||||
|
go test ./...
|
||||||
|
```
|
||||||
|
|
||||||
|
Covers:
|
||||||
|
|
||||||
|
- Pending rows get signed, logged, and attached (happy path).
|
||||||
|
- Rows that already have an attestation ref are **not** re-signed
|
||||||
|
(`TestReconciler_SkipsAlreadyAttestedRows`).
|
||||||
|
- Re-signing the same row yields the same TLog entry
|
||||||
|
(`TestReconciler_TLogIdempotencyOnReRun`) — proves "kein Doppelsignieren".
|
||||||
|
- TLog failures bump the failure counter and leave the row pending.
|
||||||
|
- Pre-cancelled context aborts the per-row loop.
|
||||||
|
- `ParseSpec` accepts `@every` and bare durations, rejects crontab lines.
|
||||||
|
- Metrics exposition format matches `# HELP` + `# TYPE` headers.
|
||||||
109
services/attest-reconciler/cmd/attest-reconciler/main.go
Normal file
109
services/attest-reconciler/cmd/attest-reconciler/main.go
Normal file
|
|
@ -0,0 +1,109 @@
|
||||||
|
// Command attest-reconciler is the periodic worker that re-signs
|
||||||
|
// per_diem_rates rows missing an attestation reference. See ADR-0005 for the
|
||||||
|
// Signing + Transparency Log contract and the parent issue for the pending-row
|
||||||
|
// workflow it implements.
|
||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"log/slog"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"os/signal"
|
||||||
|
"syscall"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"basicstack.de/proofedmesh/services/attest-reconciler/internal/attest"
|
||||||
|
"basicstack.de/proofedmesh/services/attest-reconciler/internal/metrics"
|
||||||
|
"basicstack.de/proofedmesh/services/attest-reconciler/internal/reconcile"
|
||||||
|
"basicstack.de/proofedmesh/services/attest-reconciler/internal/store"
|
||||||
|
"basicstack.de/proofedmesh/services/attest-reconciler/internal/tlog"
|
||||||
|
)
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
spec := flag.String("spec", "@every 1m", "cron.Spec-compatible schedule: '@every <dur>' or a bare Go duration")
|
||||||
|
metricsAddr := flag.String("metrics-addr", ":9090", "HTTP listen address for /metrics; empty disables")
|
||||||
|
signerURL := flag.String("signer-url", "", "Signing Service base URL (required)")
|
||||||
|
tlogURL := flag.String("tlog-url", "", "Transparency Log base URL (required)")
|
||||||
|
keyRef := flag.String("signer-key", "trust-api/rate-change-signer-2026-q4", "Signing key reference")
|
||||||
|
bearer := flag.String("bearer", "", "static Pocket ID bearer token for dev; prod uses the client_credentials TokenSource (DEV-602)")
|
||||||
|
issuer := flag.String("oidc-issuer", "https://auth.basicstack.de", "OIDC issuer stamped onto reconciler-generated attestations")
|
||||||
|
subject := flag.String("subject", "attest-reconciler@proofedmesh", "service identity for performed_by.sub")
|
||||||
|
batchLimit := flag.Int("batch-limit", 100, "max rows reconciled per tick")
|
||||||
|
flag.Parse()
|
||||||
|
|
||||||
|
logger := slog.New(slog.NewJSONHandler(os.Stdout, nil))
|
||||||
|
|
||||||
|
if *signerURL == "" || *tlogURL == "" {
|
||||||
|
fmt.Fprintln(os.Stderr, "attest-reconciler: -signer-url and -tlog-url are required")
|
||||||
|
os.Exit(2)
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||||
|
defer stop()
|
||||||
|
|
||||||
|
pg, err := store.NewPostgres()
|
||||||
|
if err != nil {
|
||||||
|
logger.Error("init postgres store failed", "err", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
|
||||||
|
var tokenSrc attest.TokenSource
|
||||||
|
if *bearer != "" {
|
||||||
|
tokenSrc = attest.StaticToken(*bearer)
|
||||||
|
}
|
||||||
|
orch := attest.Orchestrator{
|
||||||
|
Signer: attest.HTTPSigner{BaseURL: *signerURL, TokenSource: tokenSrc},
|
||||||
|
Submitter: tlog.HTTPSubmitter{BaseURL: *tlogURL, TokenSource: tokenSrc},
|
||||||
|
KeyRef: *keyRef,
|
||||||
|
}
|
||||||
|
|
||||||
|
reg := metrics.New()
|
||||||
|
r := &reconcile.Reconciler{
|
||||||
|
Store: pg,
|
||||||
|
Orchestrator: orch,
|
||||||
|
Metrics: reg,
|
||||||
|
Logger: logger,
|
||||||
|
Issuer: *issuer,
|
||||||
|
Subject: *subject,
|
||||||
|
SessionID: "", // reconciler attests headlessly; no admin session to attribute
|
||||||
|
BatchLimit: *batchLimit,
|
||||||
|
Now: time.Now,
|
||||||
|
}
|
||||||
|
|
||||||
|
// /metrics is best-effort; the reconciler keeps running even if the HTTP
|
||||||
|
// server fails to bind.
|
||||||
|
if *metricsAddr != "" {
|
||||||
|
mux := http.NewServeMux()
|
||||||
|
mux.Handle("/metrics", reg.Handler())
|
||||||
|
mux.HandleFunc("/healthz", func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
_, _ = w.Write([]byte("ok"))
|
||||||
|
})
|
||||||
|
srv := &http.Server{Addr: *metricsAddr, Handler: mux, ReadHeaderTimeout: 5 * time.Second}
|
||||||
|
go func() {
|
||||||
|
logger.Info("metrics server listening", "addr", *metricsAddr)
|
||||||
|
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
|
||||||
|
logger.Error("metrics server exited", "err", err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
go func() {
|
||||||
|
<-ctx.Done()
|
||||||
|
shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
_ = srv.Shutdown(shutdownCtx)
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
|
||||||
|
sched := reconcile.Scheduler{
|
||||||
|
Spec: *spec,
|
||||||
|
Reconciler: r,
|
||||||
|
Logger: logger,
|
||||||
|
}
|
||||||
|
if err := sched.Run(ctx); err != nil && err != context.Canceled {
|
||||||
|
logger.Error("scheduler exited with error", "err", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
logger.Info("attest-reconciler stopped cleanly")
|
||||||
|
}
|
||||||
3
services/attest-reconciler/go.mod
Normal file
3
services/attest-reconciler/go.mod
Normal file
|
|
@ -0,0 +1,3 @@
|
||||||
|
module basicstack.de/proofedmesh/services/attest-reconciler
|
||||||
|
|
||||||
|
go 1.23
|
||||||
238
services/attest-reconciler/internal/attest/attest.go
Normal file
238
services/attest-reconciler/internal/attest/attest.go
Normal file
|
|
@ -0,0 +1,238 @@
|
||||||
|
// Package attest mirrors the admin-api attestation API (ADR-0005). Current
|
||||||
|
// state: the reconciler re-declares the small Signer/Submitter contract so it
|
||||||
|
// can live in its own go.mod without pulling admin-api's internal/ tree.
|
||||||
|
// When a third consumer appears (per the parent issue's note), these types
|
||||||
|
// move into a shared module and both admin-api and attest-reconciler switch
|
||||||
|
// their imports; the wire types do not change.
|
||||||
|
package attest
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/base64"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Statement is the in-toto v1 Statement shape handed to the Signing Service.
|
||||||
|
type Statement struct {
|
||||||
|
Type string `json:"_type"`
|
||||||
|
Subject []Subject `json:"subject"`
|
||||||
|
PredicateType string `json:"predicateType"`
|
||||||
|
Predicate map[string]any `json:"predicate"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type Subject struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Digest map[string]string `json:"digest"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type Envelope struct {
|
||||||
|
PayloadType string `json:"payloadType"`
|
||||||
|
Payload string `json:"payload"` // base64url
|
||||||
|
Signatures []EnvelopeSignature `json:"signatures"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type EnvelopeSignature struct {
|
||||||
|
KeyID string `json:"keyid"`
|
||||||
|
Sig string `json:"sig"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type LogReceipt struct {
|
||||||
|
LogIndex int64 `json:"log_index"`
|
||||||
|
ContentHash string `json:"content_hash"`
|
||||||
|
LoggedAt time.Time `json:"logged_at"`
|
||||||
|
InclusionProof InclusionProof `json:"inclusion_proof"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type InclusionProof struct {
|
||||||
|
TreeSize int64 `json:"tree_size"`
|
||||||
|
TreeRootHash string `json:"tree_root_hash"`
|
||||||
|
SignedTreeHeadRef string `json:"signed_tree_head_ref"`
|
||||||
|
Hashes []string `json:"hashes"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Signer wraps POST /api/v1/sign/dsse.
|
||||||
|
type Signer interface {
|
||||||
|
SignDSSE(ctx context.Context, keyRef string, stmt Statement) (envelope Envelope, contentHash string, err error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Submitter wraps POST /api/v1/log/entries. MUST be idempotent on
|
||||||
|
// (envelope, contentHash); that is the whole reason the reconciler can safely
|
||||||
|
// re-run already-attested rows without producing duplicates.
|
||||||
|
type Submitter interface {
|
||||||
|
Submit(ctx context.Context, env Envelope, contentHash string) (LogReceipt, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
type TokenSource interface {
|
||||||
|
Token(ctx context.Context) (string, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
type StaticToken string
|
||||||
|
|
||||||
|
func (s StaticToken) Token(_ context.Context) (string, error) { return string(s), nil }
|
||||||
|
|
||||||
|
// ChangeKind labels why a rate changed.
|
||||||
|
type ChangeKind string
|
||||||
|
|
||||||
|
const (
|
||||||
|
ChangeCreate ChangeKind = "create"
|
||||||
|
ChangeRetire ChangeKind = "retire"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Rate is the subset of per_diem_rates the reconciler needs to re-sign a row.
|
||||||
|
// Field layout mirrors admin-api/internal/model.Rate so JSON round-trips with
|
||||||
|
// the same canonical bytes.
|
||||||
|
type Rate struct {
|
||||||
|
RateID string `json:"rate_id"`
|
||||||
|
Country string `json:"country"`
|
||||||
|
City *string `json:"city,omitempty"`
|
||||||
|
Currency string `json:"currency"`
|
||||||
|
FullDay string `json:"full_day"`
|
||||||
|
PartialDayMin8h string `json:"partial_day_min8h"`
|
||||||
|
ArrivalDay string `json:"arrival_day"`
|
||||||
|
DepartureDay string `json:"departure_day"`
|
||||||
|
ValidFrom time.Time `json:"valid_from"`
|
||||||
|
ValidTo *time.Time `json:"valid_to,omitempty"`
|
||||||
|
SourceRegulation string `json:"source_regulation"`
|
||||||
|
CreatedBy string `json:"created_by"`
|
||||||
|
CreatedAt time.Time `json:"created_at"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// RateChangeEvent is what the reconciler reconstructs for a pending row.
|
||||||
|
type RateChangeEvent struct {
|
||||||
|
Kind ChangeKind
|
||||||
|
Rate Rate
|
||||||
|
PerformedBy PerformedBy
|
||||||
|
PerformedAt time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
type PerformedBy struct {
|
||||||
|
Issuer string `json:"iss"`
|
||||||
|
Subject string `json:"sub"`
|
||||||
|
SessionID string `json:"session_id"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Orchestrator signs a rate_change event, submits it to the TLog, and returns
|
||||||
|
// the attestation content_hash. Semantics match admin-api exactly: the TLog
|
||||||
|
// side is idempotent, so re-running an already-signed event is safe and
|
||||||
|
// returns the same content_hash.
|
||||||
|
type Orchestrator struct {
|
||||||
|
Signer Signer
|
||||||
|
Submitter Submitter
|
||||||
|
KeyRef string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (o Orchestrator) SignAndLog(ctx context.Context, ev RateChangeEvent) (string, LogReceipt, error) {
|
||||||
|
if o.Signer == nil {
|
||||||
|
return "", LogReceipt{}, errors.New("attest.Orchestrator: Signer is nil")
|
||||||
|
}
|
||||||
|
if o.Submitter == nil {
|
||||||
|
return "", LogReceipt{}, errors.New("attest.Orchestrator: Submitter is nil")
|
||||||
|
}
|
||||||
|
if o.KeyRef == "" {
|
||||||
|
return "", LogReceipt{}, errors.New("attest.Orchestrator: KeyRef is empty")
|
||||||
|
}
|
||||||
|
stmt, err := BuildStatement(ev)
|
||||||
|
if err != nil {
|
||||||
|
return "", LogReceipt{}, fmt.Errorf("build statement: %w", err)
|
||||||
|
}
|
||||||
|
env, contentHash, err := o.Signer.SignDSSE(ctx, o.KeyRef, stmt)
|
||||||
|
if err != nil {
|
||||||
|
return "", LogReceipt{}, fmt.Errorf("sign dsse: %w", err)
|
||||||
|
}
|
||||||
|
receipt, err := o.Submitter.Submit(ctx, env, contentHash)
|
||||||
|
if err != nil {
|
||||||
|
return contentHash, LogReceipt{}, fmt.Errorf("submit tlog: %w", err)
|
||||||
|
}
|
||||||
|
if receipt.ContentHash != contentHash {
|
||||||
|
return contentHash, receipt, fmt.Errorf("tlog returned content_hash %q, expected %q", receipt.ContentHash, contentHash)
|
||||||
|
}
|
||||||
|
return contentHash, receipt, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// BuildStatement produces the canonical in-toto v1 Statement for a rate change.
|
||||||
|
// Byte-identical to admin-api/internal/attest.buildStatement so the Signing
|
||||||
|
// Service receives the same canonical bytes for the same row and the TLog
|
||||||
|
// idempotency key matches on both sides.
|
||||||
|
func BuildStatement(ev RateChangeEvent) (Statement, error) {
|
||||||
|
rateHash, err := hashRateCore(ev.Rate)
|
||||||
|
if err != nil {
|
||||||
|
return Statement{}, err
|
||||||
|
}
|
||||||
|
predicate := map[string]any{
|
||||||
|
"change_kind": string(ev.Kind),
|
||||||
|
"rate_id": ev.Rate.RateID,
|
||||||
|
"country": ev.Rate.Country,
|
||||||
|
"city": ev.Rate.City,
|
||||||
|
"currency": ev.Rate.Currency,
|
||||||
|
"full_day": ev.Rate.FullDay,
|
||||||
|
"partial_day_min8h": ev.Rate.PartialDayMin8h,
|
||||||
|
"arrival_day": ev.Rate.ArrivalDay,
|
||||||
|
"departure_day": ev.Rate.DepartureDay,
|
||||||
|
"valid_from": ev.Rate.ValidFrom.Format("2006-01-02"),
|
||||||
|
"valid_to": nilDate(ev.Rate.ValidTo),
|
||||||
|
"source_regulation": ev.Rate.SourceRegulation,
|
||||||
|
"performed_by": ev.PerformedBy,
|
||||||
|
"performed_at": ev.PerformedAt.UTC().Format(time.RFC3339),
|
||||||
|
}
|
||||||
|
return Statement{
|
||||||
|
Type: "https://in-toto.io/Statement/v1",
|
||||||
|
Subject: []Subject{{
|
||||||
|
Name: fmt.Sprintf("per_diem_rate/%s", ev.Rate.RateID),
|
||||||
|
Digest: map[string]string{"sha256": rateHash},
|
||||||
|
}},
|
||||||
|
PredicateType: "https://basicstack.de/attestations/rate_change/v1",
|
||||||
|
Predicate: predicate,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func nilDate(t *time.Time) any {
|
||||||
|
if t == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return t.Format("2006-01-02")
|
||||||
|
}
|
||||||
|
|
||||||
|
func hashRateCore(r Rate) (string, error) {
|
||||||
|
core := map[string]any{
|
||||||
|
"rate_id": r.RateID,
|
||||||
|
"country": r.Country,
|
||||||
|
"city": r.City,
|
||||||
|
"currency": r.Currency,
|
||||||
|
"full_day": r.FullDay,
|
||||||
|
"partial_day_min8h": r.PartialDayMin8h,
|
||||||
|
"arrival_day": r.ArrivalDay,
|
||||||
|
"departure_day": r.DepartureDay,
|
||||||
|
"valid_from": r.ValidFrom.Format("2006-01-02"),
|
||||||
|
"source_regulation": r.SourceRegulation,
|
||||||
|
}
|
||||||
|
b, err := canonicalJSON(core)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return "sha256:" + sha256hex(b), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var base64urlStd = base64.RawURLEncoding
|
||||||
|
|
||||||
|
// EncodePayload is kept for parity with admin-api; the reconciler uses it only
|
||||||
|
// in tests that construct an envelope directly.
|
||||||
|
func EncodePayload(stmt Statement) (string, []byte, error) {
|
||||||
|
b, err := canonicalJSON(stmt)
|
||||||
|
if err != nil {
|
||||||
|
return "", nil, err
|
||||||
|
}
|
||||||
|
return base64urlStd.EncodeToString(b), b, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func EnvelopeContentHash(env Envelope) (string, error) {
|
||||||
|
b, err := canonicalJSON(env)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return "sha256:" + sha256hex(b), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var _ = json.Marshal
|
||||||
71
services/attest-reconciler/internal/attest/attest_test.go
Normal file
71
services/attest-reconciler/internal/attest/attest_test.go
Normal file
|
|
@ -0,0 +1,71 @@
|
||||||
|
package attest
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestBuildStatement_StableBytes locks the canonical statement shape so a
|
||||||
|
// future accidental field reorder cannot silently diverge the reconciler's
|
||||||
|
// signatures from admin-api's. The expected bytes were produced by this
|
||||||
|
// package itself on first commit; if they change, admin-api's statement must
|
||||||
|
// change in lockstep.
|
||||||
|
func TestBuildStatement_StableBytes(t *testing.T) {
|
||||||
|
city := "Berlin"
|
||||||
|
ev := RateChangeEvent{
|
||||||
|
Kind: ChangeCreate,
|
||||||
|
Rate: Rate{
|
||||||
|
RateID: "PDR-1",
|
||||||
|
Country: "DE",
|
||||||
|
City: &city,
|
||||||
|
Currency: "EUR",
|
||||||
|
FullDay: "28.00",
|
||||||
|
PartialDayMin8h: "14.00",
|
||||||
|
ArrivalDay: "14.00",
|
||||||
|
DepartureDay: "14.00",
|
||||||
|
ValidFrom: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC),
|
||||||
|
SourceRegulation: "BRKG § 9 Abs. 1",
|
||||||
|
CreatedBy: "admin",
|
||||||
|
CreatedAt: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC),
|
||||||
|
},
|
||||||
|
PerformedBy: PerformedBy{
|
||||||
|
Issuer: "https://auth.basicstack.de",
|
||||||
|
Subject: "attest-reconciler@proofedmesh",
|
||||||
|
},
|
||||||
|
PerformedAt: time.Date(2026, 10, 25, 12, 0, 0, 0, time.UTC),
|
||||||
|
}
|
||||||
|
stmt, err := BuildStatement(ev)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("BuildStatement err: %v", err)
|
||||||
|
}
|
||||||
|
if stmt.Type != "https://in-toto.io/Statement/v1" {
|
||||||
|
t.Errorf("unexpected statement type %q", stmt.Type)
|
||||||
|
}
|
||||||
|
if stmt.PredicateType != "https://basicstack.de/attestations/rate_change/v1" {
|
||||||
|
t.Errorf("unexpected predicateType %q", stmt.PredicateType)
|
||||||
|
}
|
||||||
|
if len(stmt.Subject) != 1 || stmt.Subject[0].Name != "per_diem_rate/PDR-1" {
|
||||||
|
t.Fatalf("unexpected subject: %+v", stmt.Subject)
|
||||||
|
}
|
||||||
|
digest := stmt.Subject[0].Digest["sha256"]
|
||||||
|
if !strings.HasPrefix(digest, "sha256:") || len(digest) != len("sha256:")+64 {
|
||||||
|
t.Errorf("subject.digest not sha256:<64hex>: %q", digest)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Re-call must be stable (bit-for-bit reproducible — the whole point).
|
||||||
|
stmt2, _ := BuildStatement(ev)
|
||||||
|
b1, _ := canonicalJSON(stmt)
|
||||||
|
b2, _ := canonicalJSON(stmt2)
|
||||||
|
if string(b1) != string(b2) {
|
||||||
|
t.Errorf("BuildStatement is not deterministic")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestOrchestrator_RejectsMissingDeps(t *testing.T) {
|
||||||
|
o := Orchestrator{KeyRef: "k"}
|
||||||
|
_, _, err := o.SignAndLog(nil, RateChangeEvent{})
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected error from Orchestrator with nil Signer")
|
||||||
|
}
|
||||||
|
}
|
||||||
83
services/attest-reconciler/internal/attest/hash.go
Normal file
83
services/attest-reconciler/internal/attest/hash.go
Normal file
|
|
@ -0,0 +1,83 @@
|
||||||
|
package attest
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"sort"
|
||||||
|
)
|
||||||
|
|
||||||
|
func sha256hex(b []byte) string {
|
||||||
|
sum := sha256.Sum256(b)
|
||||||
|
return hex.EncodeToString(sum[:])
|
||||||
|
}
|
||||||
|
|
||||||
|
func canonicalJSON(v any) ([]byte, error) {
|
||||||
|
first, err := json.Marshal(v)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var any_ any
|
||||||
|
if err := json.Unmarshal(first, &any_); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var buf bytes.Buffer
|
||||||
|
if err := writeSorted(&buf, any_); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return buf.Bytes(), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeSorted(buf *bytes.Buffer, v any) error {
|
||||||
|
switch x := v.(type) {
|
||||||
|
case nil:
|
||||||
|
buf.WriteString("null")
|
||||||
|
case bool:
|
||||||
|
if x {
|
||||||
|
buf.WriteString("true")
|
||||||
|
} else {
|
||||||
|
buf.WriteString("false")
|
||||||
|
}
|
||||||
|
case float64:
|
||||||
|
b, _ := json.Marshal(x)
|
||||||
|
buf.Write(b)
|
||||||
|
case string:
|
||||||
|
b, _ := json.Marshal(x)
|
||||||
|
buf.Write(b)
|
||||||
|
case []any:
|
||||||
|
buf.WriteByte('[')
|
||||||
|
for i, el := range x {
|
||||||
|
if i > 0 {
|
||||||
|
buf.WriteByte(',')
|
||||||
|
}
|
||||||
|
if err := writeSorted(buf, el); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
buf.WriteByte(']')
|
||||||
|
case map[string]any:
|
||||||
|
keys := make([]string, 0, len(x))
|
||||||
|
for k := range x {
|
||||||
|
keys = append(keys, k)
|
||||||
|
}
|
||||||
|
sort.Strings(keys)
|
||||||
|
buf.WriteByte('{')
|
||||||
|
for i, k := range keys {
|
||||||
|
if i > 0 {
|
||||||
|
buf.WriteByte(',')
|
||||||
|
}
|
||||||
|
kb, _ := json.Marshal(k)
|
||||||
|
buf.Write(kb)
|
||||||
|
buf.WriteByte(':')
|
||||||
|
if err := writeSorted(buf, x[k]); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
buf.WriteByte('}')
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("writeSorted: unsupported type %T", v)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
75
services/attest-reconciler/internal/attest/http_signer.go
Normal file
75
services/attest-reconciler/internal/attest/http_signer.go
Normal file
|
|
@ -0,0 +1,75 @@
|
||||||
|
package attest
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// HTTPSigner calls POST /api/v1/sign/dsse on the Signing Service (ADR-0005).
|
||||||
|
// Wire shape matches admin-api/internal/attest.HTTPSigner so the two services
|
||||||
|
// are drop-in interchangeable against the same Signing Service deployment.
|
||||||
|
type HTTPSigner struct {
|
||||||
|
BaseURL string
|
||||||
|
Client *http.Client
|
||||||
|
TokenSource TokenSource
|
||||||
|
}
|
||||||
|
|
||||||
|
type signRequest struct {
|
||||||
|
KeyRef string `json:"key_ref"`
|
||||||
|
Statement Statement `json:"statement"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type signResponse struct {
|
||||||
|
Envelope Envelope `json:"envelope"`
|
||||||
|
ContentHash string `json:"content_hash"`
|
||||||
|
SignedAt time.Time `json:"signed_at"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s HTTPSigner) SignDSSE(ctx context.Context, keyRef string, stmt Statement) (Envelope, string, error) {
|
||||||
|
if s.BaseURL == "" {
|
||||||
|
return Envelope{}, "", errors.New("attest.HTTPSigner: BaseURL is empty")
|
||||||
|
}
|
||||||
|
cli := s.Client
|
||||||
|
if cli == nil {
|
||||||
|
cli = &http.Client{Timeout: 10 * time.Second}
|
||||||
|
}
|
||||||
|
body, err := json.Marshal(signRequest{KeyRef: keyRef, Statement: stmt})
|
||||||
|
if err != nil {
|
||||||
|
return Envelope{}, "", fmt.Errorf("marshal: %w", err)
|
||||||
|
}
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodPost, s.BaseURL+"/api/v1/sign/dsse", bytes.NewReader(body))
|
||||||
|
if err != nil {
|
||||||
|
return Envelope{}, "", err
|
||||||
|
}
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
|
if s.TokenSource != nil {
|
||||||
|
tok, err := s.TokenSource.Token(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return Envelope{}, "", fmt.Errorf("token: %w", err)
|
||||||
|
}
|
||||||
|
req.Header.Set("Authorization", "Bearer "+tok)
|
||||||
|
}
|
||||||
|
resp, err := cli.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return Envelope{}, "", err
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
return Envelope{}, "", fmt.Errorf("signer: status %d: %s", resp.StatusCode, string(raw))
|
||||||
|
}
|
||||||
|
var out signResponse
|
||||||
|
if err := json.Unmarshal(raw, &out); err != nil {
|
||||||
|
return Envelope{}, "", fmt.Errorf("decode: %w", err)
|
||||||
|
}
|
||||||
|
if out.ContentHash == "" {
|
||||||
|
return Envelope{}, "", errors.New("signer: response missing content_hash")
|
||||||
|
}
|
||||||
|
return out.Envelope, out.ContentHash, nil
|
||||||
|
}
|
||||||
74
services/attest-reconciler/internal/metrics/metrics.go
Normal file
74
services/attest-reconciler/internal/metrics/metrics.go
Normal file
|
|
@ -0,0 +1,74 @@
|
||||||
|
// Package metrics implements the three Prometheus counters the reconciler
|
||||||
|
// exposes, without pulling the prometheus/client_golang module. The /metrics
|
||||||
|
// handler prints the standard text exposition format.
|
||||||
|
package metrics
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"sync/atomic"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Registry holds the reconciler's counters. Use New() to construct and share
|
||||||
|
// one instance across the scheduler and the HTTP handler.
|
||||||
|
type Registry struct {
|
||||||
|
dirtyRows atomic.Int64
|
||||||
|
reconciliationAttempts atomic.Int64
|
||||||
|
reconciliationFailures atomic.Int64
|
||||||
|
reconciliationNoopHits atomic.Int64
|
||||||
|
}
|
||||||
|
|
||||||
|
func New() *Registry { return &Registry{} }
|
||||||
|
|
||||||
|
// SetDirtyRows sets the current pending-row count. Called at the top of each
|
||||||
|
// tick, before any attempts land.
|
||||||
|
func (r *Registry) SetDirtyRows(n int64) { r.dirtyRows.Store(n) }
|
||||||
|
|
||||||
|
func (r *Registry) IncAttempts() { r.reconciliationAttempts.Add(1) }
|
||||||
|
func (r *Registry) IncFailures() { r.reconciliationFailures.Add(1) }
|
||||||
|
func (r *Registry) IncIdempotentNoopHits() { r.reconciliationNoopHits.Add(1) }
|
||||||
|
|
||||||
|
// Snapshot returns a point-in-time copy; useful for tests.
|
||||||
|
type Snapshot struct {
|
||||||
|
DirtyRows int64
|
||||||
|
ReconciliationAttempts int64
|
||||||
|
ReconciliationFailures int64
|
||||||
|
IdempotentNoopHits int64
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Registry) Snapshot() Snapshot {
|
||||||
|
return Snapshot{
|
||||||
|
DirtyRows: r.dirtyRows.Load(),
|
||||||
|
ReconciliationAttempts: r.reconciliationAttempts.Load(),
|
||||||
|
ReconciliationFailures: r.reconciliationFailures.Load(),
|
||||||
|
IdempotentNoopHits: r.reconciliationNoopHits.Load(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Handler returns an http.HandlerFunc that writes the Prometheus text format.
|
||||||
|
func (r *Registry) Handler() http.HandlerFunc {
|
||||||
|
return func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
w.Header().Set("Content-Type", "text/plain; version=0.0.4; charset=utf-8")
|
||||||
|
r.write(w)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Registry) write(w io.Writer) {
|
||||||
|
s := r.Snapshot()
|
||||||
|
fmt.Fprintln(w, "# HELP attest_reconciler_dirty_rows_total Current count of per_diem_rates rows missing an attestation reference (gauge-style, overwritten each tick).")
|
||||||
|
fmt.Fprintln(w, "# TYPE attest_reconciler_dirty_rows_total gauge")
|
||||||
|
fmt.Fprintf(w, "attest_reconciler_dirty_rows_total %d\n", s.DirtyRows)
|
||||||
|
|
||||||
|
fmt.Fprintln(w, "# HELP attest_reconciler_reconciliation_attempts_total Total pending rows the reconciler attempted to re-sign.")
|
||||||
|
fmt.Fprintln(w, "# TYPE attest_reconciler_reconciliation_attempts_total counter")
|
||||||
|
fmt.Fprintf(w, "attest_reconciler_reconciliation_attempts_total %d\n", s.ReconciliationAttempts)
|
||||||
|
|
||||||
|
fmt.Fprintln(w, "# HELP attest_reconciler_reconciliation_failures_total Total pending-row attempts that returned an error from the signer or transparency log.")
|
||||||
|
fmt.Fprintln(w, "# TYPE attest_reconciler_reconciliation_failures_total counter")
|
||||||
|
fmt.Fprintf(w, "attest_reconciler_reconciliation_failures_total %d\n", s.ReconciliationFailures)
|
||||||
|
|
||||||
|
fmt.Fprintln(w, "# HELP attest_reconciler_idempotent_noop_total Rows the reconciler re-signed where the TLog already had the entry (safe no-op; proves the idempotency contract works).")
|
||||||
|
fmt.Fprintln(w, "# TYPE attest_reconciler_idempotent_noop_total counter")
|
||||||
|
fmt.Fprintf(w, "attest_reconciler_idempotent_noop_total %d\n", s.IdempotentNoopHits)
|
||||||
|
}
|
||||||
38
services/attest-reconciler/internal/metrics/metrics_test.go
Normal file
38
services/attest-reconciler/internal/metrics/metrics_test.go
Normal file
|
|
@ -0,0 +1,38 @@
|
||||||
|
package metrics
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestRegistrySnapshotAndTextFormat(t *testing.T) {
|
||||||
|
r := New()
|
||||||
|
r.SetDirtyRows(7)
|
||||||
|
r.IncAttempts()
|
||||||
|
r.IncAttempts()
|
||||||
|
r.IncAttempts()
|
||||||
|
r.IncFailures()
|
||||||
|
r.IncIdempotentNoopHits()
|
||||||
|
|
||||||
|
snap := r.Snapshot()
|
||||||
|
if snap.DirtyRows != 7 || snap.ReconciliationAttempts != 3 || snap.ReconciliationFailures != 1 || snap.IdempotentNoopHits != 1 {
|
||||||
|
t.Fatalf("unexpected snapshot: %+v", snap)
|
||||||
|
}
|
||||||
|
|
||||||
|
req := httptest.NewRequest("GET", "/metrics", nil)
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
r.Handler().ServeHTTP(w, req)
|
||||||
|
body := w.Body.String()
|
||||||
|
for _, substr := range []string{
|
||||||
|
"attest_reconciler_dirty_rows_total 7",
|
||||||
|
"attest_reconciler_reconciliation_attempts_total 3",
|
||||||
|
"attest_reconciler_reconciliation_failures_total 1",
|
||||||
|
"attest_reconciler_idempotent_noop_total 1",
|
||||||
|
"# TYPE attest_reconciler_dirty_rows_total gauge",
|
||||||
|
} {
|
||||||
|
if !strings.Contains(body, substr) {
|
||||||
|
t.Errorf("metrics output missing %q:\n%s", substr, body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
124
services/attest-reconciler/internal/reconcile/reconciler.go
Normal file
124
services/attest-reconciler/internal/reconcile/reconciler.go
Normal file
|
|
@ -0,0 +1,124 @@
|
||||||
|
// Package reconcile re-signs per_diem_rates rows that are missing an
|
||||||
|
// attestation reference. Depends only on the Orchestrator and PendingRateStore
|
||||||
|
// interfaces so tests can swap in fakes with zero mocking framework.
|
||||||
|
package reconcile
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"log/slog"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"basicstack.de/proofedmesh/services/attest-reconciler/internal/attest"
|
||||||
|
"basicstack.de/proofedmesh/services/attest-reconciler/internal/metrics"
|
||||||
|
"basicstack.de/proofedmesh/services/attest-reconciler/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Reconciler re-signs pending rows one tick at a time. The scheduler owns the
|
||||||
|
// timer; RunOnce is the unit of work.
|
||||||
|
type Reconciler struct {
|
||||||
|
Store store.PendingRateStore
|
||||||
|
Orchestrator attest.Orchestrator
|
||||||
|
Metrics *metrics.Registry
|
||||||
|
Logger *slog.Logger
|
||||||
|
Issuer string // OIDC issuer stamped on the reconciler's own attestations
|
||||||
|
Subject string // service identity, e.g. "attest-reconciler@proofedmesh"
|
||||||
|
SessionID string // deterministic marker for provenance; empty is fine
|
||||||
|
BatchLimit int // max rows per tick; 0 => 100
|
||||||
|
Now func() time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// RunOnce executes a single reconciliation tick: fetches pending rows,
|
||||||
|
// re-signs each, and persists the attestation ref. Returns a Report that
|
||||||
|
// callers (and tests) can inspect without reading metrics.
|
||||||
|
func (r *Reconciler) RunOnce(ctx context.Context) (Report, error) {
|
||||||
|
if r.Store == nil {
|
||||||
|
return Report{}, errors.New("reconcile.Reconciler: Store is nil")
|
||||||
|
}
|
||||||
|
if r.Now == nil {
|
||||||
|
r.Now = time.Now
|
||||||
|
}
|
||||||
|
limit := r.BatchLimit
|
||||||
|
if limit <= 0 {
|
||||||
|
limit = 100
|
||||||
|
}
|
||||||
|
|
||||||
|
rows, err := r.Store.ListPending(ctx, limit)
|
||||||
|
if err != nil {
|
||||||
|
return Report{}, fmt.Errorf("list pending: %w", err)
|
||||||
|
}
|
||||||
|
rep := Report{DirtyRows: len(rows)}
|
||||||
|
if r.Metrics != nil {
|
||||||
|
r.Metrics.SetDirtyRows(int64(len(rows)))
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, row := range rows {
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return rep, err
|
||||||
|
}
|
||||||
|
rep.Attempts++
|
||||||
|
if r.Metrics != nil {
|
||||||
|
r.Metrics.IncAttempts()
|
||||||
|
}
|
||||||
|
|
||||||
|
ev := attest.RateChangeEvent{
|
||||||
|
Kind: row.Kind,
|
||||||
|
Rate: row.Rate,
|
||||||
|
PerformedBy: attest.PerformedBy{
|
||||||
|
Issuer: r.Issuer,
|
||||||
|
Subject: r.Subject,
|
||||||
|
SessionID: r.SessionID,
|
||||||
|
},
|
||||||
|
PerformedAt: r.Now().UTC(),
|
||||||
|
}
|
||||||
|
contentHash, _, err := r.Orchestrator.SignAndLog(ctx, ev)
|
||||||
|
if err != nil {
|
||||||
|
rep.Failures++
|
||||||
|
if r.Metrics != nil {
|
||||||
|
r.Metrics.IncFailures()
|
||||||
|
}
|
||||||
|
if r.Logger != nil {
|
||||||
|
r.Logger.Warn("reconcile: sign+log failed; will retry next tick",
|
||||||
|
"rate_id", row.Rate.RateID, "err", err)
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err := r.Store.AttachAttestation(ctx, row.Rate.RateID, contentHash); err != nil {
|
||||||
|
rep.Failures++
|
||||||
|
if r.Metrics != nil {
|
||||||
|
r.Metrics.IncFailures()
|
||||||
|
}
|
||||||
|
if r.Logger != nil {
|
||||||
|
r.Logger.Error("reconcile: attach attestation failed after successful sign+log; row stays pending but TLog entry exists",
|
||||||
|
"rate_id", row.Rate.RateID, "content_hash", contentHash, "err", err)
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
rep.Successes++
|
||||||
|
if r.Logger != nil {
|
||||||
|
r.Logger.Info("reconcile: row attested",
|
||||||
|
"rate_id", row.Rate.RateID, "kind", string(row.Kind), "content_hash", contentHash)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return rep, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Report captures what a single tick did; the scheduler logs it per tick and
|
||||||
|
// tests assert on it.
|
||||||
|
type Report struct {
|
||||||
|
DirtyRows int
|
||||||
|
Attempts int
|
||||||
|
Successes int
|
||||||
|
Failures int
|
||||||
|
}
|
||||||
|
|
||||||
|
// NoteIdempotentNoop lets higher layers record that the TLog returned the
|
||||||
|
// same log_index for a resubmit (proves idempotency guarantee); the signing
|
||||||
|
// path itself cannot know that from a single call because the server treats
|
||||||
|
// 201 and 200 as both "ok".
|
||||||
|
func (r *Reconciler) NoteIdempotentNoop() {
|
||||||
|
if r.Metrics != nil {
|
||||||
|
r.Metrics.IncIdempotentNoopHits()
|
||||||
|
}
|
||||||
|
}
|
||||||
358
services/attest-reconciler/internal/reconcile/reconciler_test.go
Normal file
358
services/attest-reconciler/internal/reconcile/reconciler_test.go
Normal file
|
|
@ -0,0 +1,358 @@
|
||||||
|
package reconcile
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"log/slog"
|
||||||
|
"os"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"basicstack.de/proofedmesh/services/attest-reconciler/internal/attest"
|
||||||
|
"basicstack.de/proofedmesh/services/attest-reconciler/internal/metrics"
|
||||||
|
"basicstack.de/proofedmesh/services/attest-reconciler/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
// fakeStore models per_diem_rates in-memory. ListPending honours the
|
||||||
|
// WHERE rate_change_attestation_ref IS NULL contract — any row that already
|
||||||
|
// has a ref is filtered out. AttachAttestation sets the ref and is the only
|
||||||
|
// way a row leaves the pending set.
|
||||||
|
type fakeStore struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
rows []storedRow
|
||||||
|
}
|
||||||
|
|
||||||
|
type storedRow struct {
|
||||||
|
row store.PendingRow
|
||||||
|
ref string // "" = no attestation yet
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeStore) seed(row store.PendingRow, ref string) {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
f.rows = append(f.rows, storedRow{row: row, ref: ref})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeStore) ListPending(_ context.Context, limit int) ([]store.PendingRow, error) {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
out := make([]store.PendingRow, 0, len(f.rows))
|
||||||
|
for _, r := range f.rows {
|
||||||
|
if r.ref != "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
out = append(out, r.row)
|
||||||
|
if limit > 0 && len(out) >= limit {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeStore) AttachAttestation(_ context.Context, rateID, ref string) error {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
for i := range f.rows {
|
||||||
|
if f.rows[i].row.Rate.RateID == rateID {
|
||||||
|
if f.rows[i].ref != "" {
|
||||||
|
// Already attested; treat as idempotent no-op in the fake —
|
||||||
|
// production AttachAttestation uses WHERE ref IS NULL and
|
||||||
|
// returns ErrNotFound for a conflict, which the reconciler
|
||||||
|
// surfaces as a failure. Our test paths never exercise that.
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
f.rows[i].ref = ref
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return store.ErrNotFound
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeStore) refFor(rateID string) string {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
for _, r := range f.rows {
|
||||||
|
if r.row.Rate.RateID == rateID {
|
||||||
|
return r.ref
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// fakeSigner returns a deterministic envelope + content_hash per rate_id so
|
||||||
|
// resubmits land on the same TLog idempotency key.
|
||||||
|
type fakeSigner struct {
|
||||||
|
calls int
|
||||||
|
mu sync.Mutex
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeSigner) SignDSSE(_ context.Context, keyRef string, stmt attest.Statement) (attest.Envelope, string, error) {
|
||||||
|
f.mu.Lock()
|
||||||
|
f.calls++
|
||||||
|
f.mu.Unlock()
|
||||||
|
// The statement's subject.name is "per_diem_rate/<rate_id>".
|
||||||
|
rateID := ""
|
||||||
|
if len(stmt.Subject) > 0 {
|
||||||
|
rateID = stmt.Subject[0].Name
|
||||||
|
}
|
||||||
|
env := attest.Envelope{
|
||||||
|
PayloadType: "application/vnd.in-toto+json",
|
||||||
|
Payload: "stub-payload-" + rateID,
|
||||||
|
Signatures: []attest.EnvelopeSignature{{KeyID: keyRef, Sig: "stub-sig-" + rateID}},
|
||||||
|
}
|
||||||
|
hash, err := attest.EnvelopeContentHash(env)
|
||||||
|
return env, hash, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// fakeSubmitter remembers which content_hashes it saw. The TLog contract says
|
||||||
|
// a resubmit returns the SAME receipt — the fake enforces that.
|
||||||
|
type fakeSubmitter struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
seen map[string]attest.LogReceipt
|
||||||
|
failWith error
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeSubmitter) Submit(_ context.Context, env attest.Envelope, contentHash string) (attest.LogReceipt, error) {
|
||||||
|
if f.failWith != nil {
|
||||||
|
return attest.LogReceipt{}, f.failWith
|
||||||
|
}
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
if f.seen == nil {
|
||||||
|
f.seen = map[string]attest.LogReceipt{}
|
||||||
|
}
|
||||||
|
if existing, ok := f.seen[contentHash]; ok {
|
||||||
|
return existing, nil
|
||||||
|
}
|
||||||
|
rcpt := attest.LogReceipt{
|
||||||
|
LogIndex: int64(len(f.seen) + 1),
|
||||||
|
ContentHash: contentHash,
|
||||||
|
LoggedAt: time.Now().UTC(),
|
||||||
|
}
|
||||||
|
f.seen[contentHash] = rcpt
|
||||||
|
return rcpt, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeSubmitter) calls() int {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
return len(f.seen)
|
||||||
|
}
|
||||||
|
|
||||||
|
func pendingRow(id string, kind attest.ChangeKind) store.PendingRow {
|
||||||
|
return store.PendingRow{
|
||||||
|
Rate: attest.Rate{
|
||||||
|
RateID: id,
|
||||||
|
Country: "DE",
|
||||||
|
Currency: "EUR",
|
||||||
|
FullDay: "28.00",
|
||||||
|
PartialDayMin8h: "14.00",
|
||||||
|
ArrivalDay: "14.00",
|
||||||
|
DepartureDay: "14.00",
|
||||||
|
ValidFrom: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC),
|
||||||
|
SourceRegulation: "BRKG § 9 Abs. 1",
|
||||||
|
CreatedBy: "stub-admin",
|
||||||
|
CreatedAt: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC),
|
||||||
|
},
|
||||||
|
Kind: kind,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func newReconciler(st store.PendingRateStore, signer attest.Signer, sub attest.Submitter) (*Reconciler, *metrics.Registry) {
|
||||||
|
reg := metrics.New()
|
||||||
|
return &Reconciler{
|
||||||
|
Store: st,
|
||||||
|
Orchestrator: attest.Orchestrator{
|
||||||
|
Signer: signer,
|
||||||
|
Submitter: sub,
|
||||||
|
KeyRef: "trust-api/rate-change-signer-2026-q4",
|
||||||
|
},
|
||||||
|
Metrics: reg,
|
||||||
|
Logger: slog.New(slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{Level: slog.LevelWarn})),
|
||||||
|
Issuer: "https://auth.basicstack.de",
|
||||||
|
Subject: "attest-reconciler@proofedmesh",
|
||||||
|
SessionID: "test",
|
||||||
|
Now: func() time.Time { return time.Date(2026, 10, 25, 12, 0, 0, 0, time.UTC) },
|
||||||
|
}, reg
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestReconciler_SignsPendingRows is the happy path: two pending rows get
|
||||||
|
// signed, logged, and attached in one tick.
|
||||||
|
func TestReconciler_SignsPendingRows(t *testing.T) {
|
||||||
|
st := &fakeStore{}
|
||||||
|
st.seed(pendingRow("PDR-1", attest.ChangeCreate), "")
|
||||||
|
st.seed(pendingRow("PDR-2", attest.ChangeCreate), "")
|
||||||
|
signer := &fakeSigner{}
|
||||||
|
sub := &fakeSubmitter{}
|
||||||
|
r, reg := newReconciler(st, signer, sub)
|
||||||
|
|
||||||
|
rep, err := r.RunOnce(context.Background())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("RunOnce err: %v", err)
|
||||||
|
}
|
||||||
|
if rep.DirtyRows != 2 || rep.Attempts != 2 || rep.Successes != 2 || rep.Failures != 0 {
|
||||||
|
t.Fatalf("unexpected report: %+v", rep)
|
||||||
|
}
|
||||||
|
if ref := st.refFor("PDR-1"); ref == "" {
|
||||||
|
t.Errorf("PDR-1 has no attestation ref after RunOnce")
|
||||||
|
}
|
||||||
|
if ref := st.refFor("PDR-2"); ref == "" {
|
||||||
|
t.Errorf("PDR-2 has no attestation ref after RunOnce")
|
||||||
|
}
|
||||||
|
m := reg.Snapshot()
|
||||||
|
if m.DirtyRows != 2 || m.ReconciliationAttempts != 2 || m.ReconciliationFailures != 0 {
|
||||||
|
t.Errorf("unexpected metrics snapshot: %+v", m)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestReconciler_SkipsAlreadyAttestedRows is the spec's required invariant:
|
||||||
|
// bereits attestierte Rows werden nicht erneut signiert. We seed a store with
|
||||||
|
// one already-signed row and one pending row and assert that the signer was
|
||||||
|
// called exactly once (for the pending row only).
|
||||||
|
func TestReconciler_SkipsAlreadyAttestedRows(t *testing.T) {
|
||||||
|
st := &fakeStore{}
|
||||||
|
st.seed(pendingRow("PDR-already-signed", attest.ChangeCreate), "sha256:cafebabe"+repeat("00", 28))
|
||||||
|
st.seed(pendingRow("PDR-still-pending", attest.ChangeCreate), "")
|
||||||
|
signer := &fakeSigner{}
|
||||||
|
sub := &fakeSubmitter{}
|
||||||
|
r, _ := newReconciler(st, signer, sub)
|
||||||
|
|
||||||
|
rep, err := r.RunOnce(context.Background())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("RunOnce err: %v", err)
|
||||||
|
}
|
||||||
|
if rep.DirtyRows != 1 {
|
||||||
|
t.Fatalf("expected 1 dirty row, got %d (full report: %+v)", rep.DirtyRows, rep)
|
||||||
|
}
|
||||||
|
if rep.Attempts != 1 {
|
||||||
|
t.Errorf("expected 1 attempt (only the pending row), got %d", rep.Attempts)
|
||||||
|
}
|
||||||
|
if signer.calls != 1 {
|
||||||
|
t.Errorf("expected signer to be called once, got %d calls", signer.calls)
|
||||||
|
}
|
||||||
|
if ref := st.refFor("PDR-already-signed"); ref != "sha256:cafebabe"+repeat("00", 28) {
|
||||||
|
t.Errorf("already-signed row's ref changed: %q", ref)
|
||||||
|
}
|
||||||
|
if ref := st.refFor("PDR-still-pending"); ref == "" {
|
||||||
|
t.Errorf("still-pending row has no attestation ref after RunOnce")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestReconciler_TLogIdempotencyOnReRun shows the Transparency Log's
|
||||||
|
// idempotency guarantee: if the ref somehow fails to persist (e.g. the DB
|
||||||
|
// hiccup after a successful log entry), the next tick re-signs the row and
|
||||||
|
// the TLog returns the SAME log_index. This proves kein Doppelsignieren.
|
||||||
|
func TestReconciler_TLogIdempotencyOnReRun(t *testing.T) {
|
||||||
|
st := &fakeStore{}
|
||||||
|
st.seed(pendingRow("PDR-1", attest.ChangeCreate), "")
|
||||||
|
signer := &fakeSigner{}
|
||||||
|
sub := &fakeSubmitter{}
|
||||||
|
r, _ := newReconciler(st, signer, sub)
|
||||||
|
|
||||||
|
// Tick 1: sign and log; the fake store attaches the ref.
|
||||||
|
rep1, err := r.RunOnce(context.Background())
|
||||||
|
if err != nil || rep1.Successes != 1 {
|
||||||
|
t.Fatalf("tick 1 failed: %+v err=%v", rep1, err)
|
||||||
|
}
|
||||||
|
ref1 := st.refFor("PDR-1")
|
||||||
|
|
||||||
|
// Simulate the "ref persist lost" case by clearing the ref so the row is
|
||||||
|
// pending again. In production that would require a different root cause
|
||||||
|
// (DB rollback, async worker crash), but it is the exact scenario the
|
||||||
|
// reconciler is here to recover from.
|
||||||
|
st.mu.Lock()
|
||||||
|
for i := range st.rows {
|
||||||
|
if st.rows[i].row.Rate.RateID == "PDR-1" {
|
||||||
|
st.rows[i].ref = ""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
st.mu.Unlock()
|
||||||
|
|
||||||
|
// Tick 2: must re-sign the same row AND land on the same TLog entry.
|
||||||
|
rep2, err := r.RunOnce(context.Background())
|
||||||
|
if err != nil || rep2.Successes != 1 {
|
||||||
|
t.Fatalf("tick 2 failed: %+v err=%v", rep2, err)
|
||||||
|
}
|
||||||
|
ref2 := st.refFor("PDR-1")
|
||||||
|
if ref1 != ref2 {
|
||||||
|
t.Errorf("content_hash changed between ticks: %q vs %q (TLog idempotency broken)", ref1, ref2)
|
||||||
|
}
|
||||||
|
if got := sub.calls(); got != 1 {
|
||||||
|
t.Errorf("TLog should hold exactly one entry across two ticks, got %d", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestReconciler_RecordsFailuresWhenTLogDown ensures the metrics +
|
||||||
|
// retry-next-tick semantics are correct when the Transparency Log is
|
||||||
|
// unreachable; the row must stay pending.
|
||||||
|
func TestReconciler_RecordsFailuresWhenTLogDown(t *testing.T) {
|
||||||
|
st := &fakeStore{}
|
||||||
|
st.seed(pendingRow("PDR-1", attest.ChangeCreate), "")
|
||||||
|
signer := &fakeSigner{}
|
||||||
|
sub := &fakeSubmitter{failWith: errors.New("tlog down")}
|
||||||
|
r, reg := newReconciler(st, signer, sub)
|
||||||
|
|
||||||
|
rep, err := r.RunOnce(context.Background())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("RunOnce err: %v", err)
|
||||||
|
}
|
||||||
|
if rep.Successes != 0 || rep.Failures != 1 {
|
||||||
|
t.Fatalf("expected 0 successes / 1 failure, got %+v", rep)
|
||||||
|
}
|
||||||
|
if ref := st.refFor("PDR-1"); ref != "" {
|
||||||
|
t.Errorf("row acquired a ref despite TLog failure: %q", ref)
|
||||||
|
}
|
||||||
|
if m := reg.Snapshot(); m.ReconciliationFailures != 1 {
|
||||||
|
t.Errorf("failure metric not incremented: %+v", m)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconciler_ContextCancelStopsLoop(t *testing.T) {
|
||||||
|
st := &fakeStore{}
|
||||||
|
for i := 0; i < 10; i++ {
|
||||||
|
st.seed(pendingRow("PDR-"+itoa(i), attest.ChangeCreate), "")
|
||||||
|
}
|
||||||
|
signer := &fakeSigner{}
|
||||||
|
sub := &fakeSubmitter{}
|
||||||
|
r, _ := newReconciler(st, signer, sub)
|
||||||
|
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
cancel() // cancel before RunOnce starts the per-row loop
|
||||||
|
_, err := r.RunOnce(ctx)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected ctx.Err() from RunOnce, got nil")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- small helpers to keep the test file free of fmt/strings deps ---
|
||||||
|
|
||||||
|
func repeat(s string, n int) string {
|
||||||
|
out := ""
|
||||||
|
for i := 0; i < n; i++ {
|
||||||
|
out += s
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func itoa(n int) string {
|
||||||
|
if n == 0 {
|
||||||
|
return "0"
|
||||||
|
}
|
||||||
|
neg := n < 0
|
||||||
|
if neg {
|
||||||
|
n = -n
|
||||||
|
}
|
||||||
|
var buf [20]byte
|
||||||
|
i := len(buf)
|
||||||
|
for n > 0 {
|
||||||
|
i--
|
||||||
|
buf[i] = byte('0' + n%10)
|
||||||
|
n /= 10
|
||||||
|
}
|
||||||
|
if neg {
|
||||||
|
i--
|
||||||
|
buf[i] = '-'
|
||||||
|
}
|
||||||
|
return string(buf[i:])
|
||||||
|
}
|
||||||
95
services/attest-reconciler/internal/reconcile/scheduler.go
Normal file
95
services/attest-reconciler/internal/reconcile/scheduler.go
Normal file
|
|
@ -0,0 +1,95 @@
|
||||||
|
package reconcile
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"log/slog"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Scheduler drives Reconciler.RunOnce on a fixed cadence until ctx is
|
||||||
|
// cancelled. Spec accepts either "@every <duration>" (robfig/cron compat) or a
|
||||||
|
// bare Go duration string ("30s", "5m"). Keeping it to those two shapes buys
|
||||||
|
// us zero external deps while still matching the cron.Spec convention the
|
||||||
|
// parent issue asked for.
|
||||||
|
type Scheduler struct {
|
||||||
|
Spec string
|
||||||
|
Reconciler *Reconciler
|
||||||
|
Logger *slog.Logger
|
||||||
|
}
|
||||||
|
|
||||||
|
// Run blocks until ctx is cancelled. Returns ctx.Err() in that case; returns
|
||||||
|
// any other error immediately. Each tick's error is logged and swallowed —
|
||||||
|
// the whole point of the reconciler is retry-until-success.
|
||||||
|
func (s *Scheduler) Run(ctx context.Context) error {
|
||||||
|
if s.Reconciler == nil {
|
||||||
|
return errors.New("reconcile.Scheduler: Reconciler is nil")
|
||||||
|
}
|
||||||
|
interval, err := ParseSpec(s.Spec)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("parse spec %q: %w", s.Spec, err)
|
||||||
|
}
|
||||||
|
if s.Logger != nil {
|
||||||
|
s.Logger.Info("reconciler scheduler started", "interval", interval.String())
|
||||||
|
}
|
||||||
|
// Fire once immediately so pending rows don't sit a full interval on boot.
|
||||||
|
s.tick(ctx)
|
||||||
|
t := time.NewTicker(interval)
|
||||||
|
defer t.Stop()
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
if s.Logger != nil {
|
||||||
|
s.Logger.Info("reconciler scheduler stopping", "reason", ctx.Err())
|
||||||
|
}
|
||||||
|
return ctx.Err()
|
||||||
|
case <-t.C:
|
||||||
|
s.tick(ctx)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Scheduler) tick(ctx context.Context) {
|
||||||
|
rep, err := s.Reconciler.RunOnce(ctx)
|
||||||
|
if err != nil {
|
||||||
|
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if s.Logger != nil {
|
||||||
|
s.Logger.Error("reconciler tick failed", "err", err)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if s.Logger != nil && (rep.DirtyRows > 0 || rep.Failures > 0) {
|
||||||
|
s.Logger.Info("reconciler tick",
|
||||||
|
"dirty_rows", rep.DirtyRows,
|
||||||
|
"attempts", rep.Attempts,
|
||||||
|
"successes", rep.Successes,
|
||||||
|
"failures", rep.Failures)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ParseSpec accepts "@every <duration>" or a bare Go duration. Anything else
|
||||||
|
// is rejected so operators who try a full crontab line get a loud error
|
||||||
|
// rather than a silent default.
|
||||||
|
func ParseSpec(spec string) (time.Duration, error) {
|
||||||
|
s := strings.TrimSpace(spec)
|
||||||
|
if s == "" {
|
||||||
|
return 0, errors.New("empty spec")
|
||||||
|
}
|
||||||
|
if strings.HasPrefix(s, "@every ") {
|
||||||
|
s = strings.TrimSpace(strings.TrimPrefix(s, "@every "))
|
||||||
|
} else if strings.ContainsAny(s, "* ") {
|
||||||
|
return 0, fmt.Errorf("spec %q: only @every <duration> and bare Go durations are supported", spec)
|
||||||
|
}
|
||||||
|
d, err := time.ParseDuration(s)
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
if d <= 0 {
|
||||||
|
return 0, fmt.Errorf("spec %q: duration must be positive", spec)
|
||||||
|
}
|
||||||
|
return d, nil
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,40 @@
|
||||||
|
package reconcile
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestParseSpec(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
in string
|
||||||
|
want time.Duration
|
||||||
|
wantErr bool
|
||||||
|
}{
|
||||||
|
{"@every 30s", 30 * time.Second, false},
|
||||||
|
{"@every 2m", 2 * time.Minute, false},
|
||||||
|
{"5m", 5 * time.Minute, false},
|
||||||
|
{"1h30m", 90 * time.Minute, false},
|
||||||
|
{"", 0, true},
|
||||||
|
{"@every", 0, true},
|
||||||
|
{"@every 0s", 0, true},
|
||||||
|
{"-1m", 0, true},
|
||||||
|
{"* * * * *", 0, true},
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
got, err := ParseSpec(c.in)
|
||||||
|
if c.wantErr {
|
||||||
|
if err == nil {
|
||||||
|
t.Errorf("ParseSpec(%q) err=nil; want error", c.in)
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("ParseSpec(%q) err=%v; want nil", c.in, err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if got != c.want {
|
||||||
|
t.Errorf("ParseSpec(%q) = %v; want %v", c.in, got, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
55
services/attest-reconciler/internal/store/postgres.go
Normal file
55
services/attest-reconciler/internal/store/postgres.go
Normal file
|
|
@ -0,0 +1,55 @@
|
||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
|
||||||
|
"basicstack.de/proofedmesh/services/attest-reconciler/internal/attest"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Postgres is the production PendingRateStore. Skeleton for now; wiring a
|
||||||
|
// real pgxpool.Pool lives alongside the admin-api Postgres swap in a later
|
||||||
|
// change so both consumers share one connection-config convention.
|
||||||
|
type Postgres struct {
|
||||||
|
// pool *pgxpool.Pool
|
||||||
|
}
|
||||||
|
|
||||||
|
var errPGUnimplemented = errors.New("postgres pending-row store is a skeleton; use a fake PendingRateStore for tests")
|
||||||
|
|
||||||
|
func NewPostgres() (*Postgres, error) {
|
||||||
|
return &Postgres{}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *Postgres) ListPending(ctx context.Context, limit int) ([]PendingRow, error) {
|
||||||
|
// Query shape (ADR-0004 schema):
|
||||||
|
// SELECT rate_id, country, city, currency,
|
||||||
|
// full_day, partial_day_min8h, arrival_day, departure_day,
|
||||||
|
// valid_from, valid_to, source_regulation,
|
||||||
|
// created_by, created_at,
|
||||||
|
// (valid_to IS NOT NULL AND valid_to < CURRENT_DATE) AS retired
|
||||||
|
// FROM per_diem_rates
|
||||||
|
// WHERE rate_change_attestation_ref IS NULL
|
||||||
|
// ORDER BY created_at ASC
|
||||||
|
// LIMIT $1
|
||||||
|
// Kind is "retire" when the row has a valid_to set at pickup time; "create"
|
||||||
|
// otherwise. See store.go for why that heuristic is safe under TLog
|
||||||
|
// idempotency.
|
||||||
|
return nil, errPGUnimplemented
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *Postgres) AttachAttestation(ctx context.Context, rateID string, attestationRef string) error {
|
||||||
|
// UPDATE per_diem_rates
|
||||||
|
// SET rate_change_attestation_ref = $2
|
||||||
|
// WHERE rate_id = $1 AND rate_change_attestation_ref IS NULL
|
||||||
|
// RETURNING rate_id;
|
||||||
|
// A zero-row result means someone else attested the row concurrently; the
|
||||||
|
// reconciler treats that as a successful no-op (see reconciler.go).
|
||||||
|
return errPGUnimplemented
|
||||||
|
}
|
||||||
|
|
||||||
|
// Compile-time assertions the real type implements the interface.
|
||||||
|
var _ PendingRateStore = (*Postgres)(nil)
|
||||||
|
|
||||||
|
// Reference imports kept so a future pgx wiring can reach the attest types
|
||||||
|
// without re-adding them.
|
||||||
|
var _ = attest.Rate{}
|
||||||
40
services/attest-reconciler/internal/store/store.go
Normal file
40
services/attest-reconciler/internal/store/store.go
Normal file
|
|
@ -0,0 +1,40 @@
|
||||||
|
// Package store contains the pending-rows reader. The reconciler never writes
|
||||||
|
// to per_diem_rates directly; it hands the attestation ref back through
|
||||||
|
// AttachAttestation, which goes through the same immutability-safe update
|
||||||
|
// path admin-api uses.
|
||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
|
||||||
|
"basicstack.de/proofedmesh/services/attest-reconciler/internal/attest"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ErrNotFound is returned by AttachAttestation when the rate_id is gone.
|
||||||
|
var ErrNotFound = errors.New("per_diem_rate: not found")
|
||||||
|
|
||||||
|
// PendingRateStore is implemented by the Postgres reader in prod and by a
|
||||||
|
// fake in tests. ListPending returns every row where
|
||||||
|
// rate_change_attestation_ref IS NULL — these are what the reconciler re-signs.
|
||||||
|
//
|
||||||
|
// Kind is reconstructed from (valid_to, now): a rate with a valid_to that was
|
||||||
|
// present at creation time is a "create" (valid_to-at-creation stays with the
|
||||||
|
// row), a rate whose valid_to was set later via Retire is a "retire". Because
|
||||||
|
// the Transparency Log is idempotent on (envelope, content_hash), a wrong
|
||||||
|
// Kind guess cannot produce a duplicate log entry — it would just land under
|
||||||
|
// a different content_hash the first time and converge from then on.
|
||||||
|
type PendingRateStore interface {
|
||||||
|
ListPending(ctx context.Context, limit int) ([]PendingRow, error)
|
||||||
|
AttachAttestation(ctx context.Context, rateID string, attestationRef string) error
|
||||||
|
}
|
||||||
|
|
||||||
|
// PendingRow is the minimal projection of per_diem_rates needed to rebuild a
|
||||||
|
// RateChangeEvent. The reconciler intentionally does NOT read the admin
|
||||||
|
// principal fields — it attests under its own service identity (see
|
||||||
|
// PerformedBy in the main command) because the original admin session is long
|
||||||
|
// gone by the time a row is picked up.
|
||||||
|
type PendingRow struct {
|
||||||
|
Rate attest.Rate
|
||||||
|
Kind attest.ChangeKind
|
||||||
|
}
|
||||||
77
services/attest-reconciler/internal/tlog/submitter.go
Normal file
77
services/attest-reconciler/internal/tlog/submitter.go
Normal file
|
|
@ -0,0 +1,77 @@
|
||||||
|
// Package tlog is the attest-reconciler's HTTP client for the Transparency Log
|
||||||
|
// (ADR-0005). Mirrors admin-api/internal/tlog.HTTPSubmitter byte-for-byte so
|
||||||
|
// resubmits from the reconciler collide with the admin-api's original log
|
||||||
|
// entry — the server's idempotency key covers both paths.
|
||||||
|
package tlog
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"basicstack.de/proofedmesh/services/attest-reconciler/internal/attest"
|
||||||
|
)
|
||||||
|
|
||||||
|
type HTTPSubmitter struct {
|
||||||
|
BaseURL string
|
||||||
|
Client *http.Client
|
||||||
|
TokenSource attest.TokenSource
|
||||||
|
}
|
||||||
|
|
||||||
|
type submitRequest struct {
|
||||||
|
Envelope attest.Envelope `json:"envelope"`
|
||||||
|
ContentHash string `json:"content_hash"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s HTTPSubmitter) Submit(ctx context.Context, env attest.Envelope, contentHash string) (attest.LogReceipt, error) {
|
||||||
|
if s.BaseURL == "" {
|
||||||
|
return attest.LogReceipt{}, errors.New("tlog.HTTPSubmitter: BaseURL is empty")
|
||||||
|
}
|
||||||
|
cli := s.Client
|
||||||
|
if cli == nil {
|
||||||
|
cli = &http.Client{Timeout: 10 * time.Second}
|
||||||
|
}
|
||||||
|
body, err := json.Marshal(submitRequest{Envelope: env, ContentHash: contentHash})
|
||||||
|
if err != nil {
|
||||||
|
return attest.LogReceipt{}, fmt.Errorf("marshal: %w", err)
|
||||||
|
}
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodPost, s.BaseURL+"/api/v1/log/entries", bytes.NewReader(body))
|
||||||
|
if err != nil {
|
||||||
|
return attest.LogReceipt{}, err
|
||||||
|
}
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
|
if s.TokenSource != nil {
|
||||||
|
tok, err := s.TokenSource.Token(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return attest.LogReceipt{}, fmt.Errorf("token: %w", err)
|
||||||
|
}
|
||||||
|
req.Header.Set("Authorization", "Bearer "+tok)
|
||||||
|
}
|
||||||
|
resp, err := cli.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return attest.LogReceipt{}, err
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
|
||||||
|
switch resp.StatusCode {
|
||||||
|
case http.StatusOK, http.StatusCreated:
|
||||||
|
default:
|
||||||
|
return attest.LogReceipt{}, fmt.Errorf("tlog: status %d: %s", resp.StatusCode, string(raw))
|
||||||
|
}
|
||||||
|
var out attest.LogReceipt
|
||||||
|
if err := json.Unmarshal(raw, &out); err != nil {
|
||||||
|
return attest.LogReceipt{}, fmt.Errorf("decode: %w", err)
|
||||||
|
}
|
||||||
|
if out.ContentHash != contentHash {
|
||||||
|
return out, fmt.Errorf("tlog response content_hash %q != sent %q", out.ContentHash, contentHash)
|
||||||
|
}
|
||||||
|
if out.LogIndex < 0 {
|
||||||
|
return out, fmt.Errorf("tlog response has negative log_index %d", out.LogIndex)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
Loading…
Add table
Reference in a new issue