Overlay Server

A reference implementation of an overlay node built on @bsv/overlay-express. Implements topic managers and lookup services to enable distributed applications to organize and query blockchain data efficiently.

What it does

The Overlay Server bootstraps topic managers and lookup services from @bsv/overlay-express. Topic managers validate which transaction outputs are admissible to the overlay by decoding PushDrop-encoded outputs, verifying signatures and cryptographic proofs, and returning AdmittanceInstructions. Lookup services store admitted outputs in MongoDB and respond to queries via the SLAP protocol. The server coordinates with a WalletAdvertiser for overlay advertising and connects to both MongoDB (lookup storage) and MySQL/Knex (overlay transaction tracking).

Clients submit transaction outputs via HTTP, the server routes valid outputs through registered topic managers, stores admitted outputs, and serves queries from any peer.

When to deploy this

• Running an overlay node with topic managers and lookup services
• You need to organize and index blockchain outputs by topic
• Require distributed query capability across peers
• Building services on top of overlay (ProtoMap, CertMap, UHRP, etc.)

Dependencies

HTTP endpoints

Additional endpoints exposed by configured topic managers and lookup services (see src/services/ for ProtoMap, CertMap, BasketMap, UHRP, Identity, MessageBox, UMP, etc.).

The UMP manager and lookup service share the Mongo-backed ump_identity_reservations collection. The first current UTXO to claim a presentation or recovery hash owns that value until a transaction consumes the outpoint. Back up MongoDB before introducing the collection. Existing ambiguous UMP rows stay queryable for lineage/WAB-pin recovery; the migration does not delete them.

The infrastructure source capability-gates this wiring so pull-request image contracts can still build against the latest published package. A production image does not enable cross-replica reservations until the protected package release publishes @bsv/overlay-topics 1.7.0 and the generated dependency-sync PR pins that version in infra/overlay-server.

WebSocket endpoints

None (HTTP-only OverlayExpress endpoints).

Configuration (env vars)

At least one transaction propagation provider must be configured: ARCADE_URL or ARC_API_KEY. Production deployments should prefer Arcade-first plus Arc fallback when both are available.

Run locally

# Install dependencies
npm install

# Development with hot-reload (uses tsx)
npm run dev

# Build TypeScript to dist/
npm run build

# Run production build
npm start

# Full stack with Docker Compose (app + MongoDB + MySQL)
docker compose up --build

Deploy to production

# Multi-stage build: Node builder → production runtime
docker build -t overlay-server:latest .

# Run with environment variables
docker run -d \
  -e NODE_NAME=overlay-node-1 \
  -e SERVER_PRIVATE_KEY=<32-byte-hex> \
  -e HOSTING_URL=https://overlay.example.com \
  -e ADMIN_TOKEN=<secure-token> \
  -e WALLET_STORAGE_URL=https://store-us-1.bsvb.tech \
  -e NETWORK=main \
  -e ARCADE_URL=https://arcade-v2-us-1.bsvblockchain.tech \
  -e ARCADE_API_KEY=<arcade-key> \
  -e ARCADE_DEPLOYMENT_ID=overlay-node-1 \
  -e ARC_API_KEY=<arc-key> \
  -e ARC_CALLBACK_TOKEN=<callback-token> \
  -e USE_ARCADE_CHAINTRACKS=true \
  -e BASM_ENABLED=true \
  -e UNPROVEN_EVICTION_BLOCKS=144 \
  -e UNPROVEN_MAINTENANCE_INTERVAL_MS=3600000 \
  -e MONGO_URL=mongodb://mongo:27017/overlay \
  -e KNEX_URL=mysql://user:pass@mysql:3306/overlay \
  -p 8080:8080 \
  overlay-server:latest

# Or with Docker Compose (includes MongoDB, MySQL, janitor service)
docker compose up -d

Service listens on port 8080 by default. Kubernetes deployment files available in deploy/ (app-deployment.yaml, mongodb, mysql with persistent volumes).

Migrations

Managed by @bsv/overlay-express and Knex. Auto-run on startup. Tables: outputs, topic_managers, lookup_services, with indexes on identity_key, output_hash, blockchain_height.

Provider and maintenance behavior

The deployment wrapper configures providers in this order:

1. Arcade, when ARCADE_URL is set.
2. Arc fallback, when ARC_API_KEY is set.
3. Chaintracks header validation and reorg support from CHAINTRACKS_URL, or from ARCADE_URL when USE_ARCADE_CHAINTRACKS=true.

THROW_ON_BROADCAST_FAIL=true is the production default. With that setting, a submitted transaction is not committed to overlay state unless a provider accepts it or reports it already known. Transient provider errors should be retried by the client or caller; terminal provider errors are not treated as successful overlay admission.

Provider callbacks posted to /arc-ingest are used to attach proofs or mark terminal outcomes. Double-spend or terminal invalid callbacks evict the transaction from admitted overlay state and notify lookup services through the normal eviction path.

When UNPROVEN_MAINTENANCE_INTERVAL_MS is greater than zero, the server periodically runs proof refresh followed by unproven eviction. The same behavior is available manually through POST /admin/maintainUnproven.

Health checks

Use:

• GET /health/live for Kubernetes liveness.
• GET /health/ready for readiness.
• GET /health for operator diagnostics.

The full health response includes provider context showing whether Arc, Arcade, and Chaintracks are configured, plus BASM and unproven-maintenance settings. Alert on readiness failures, repeated provider callback errors, repeated overlay.unproven_maintenance failures, and increasing unproven transaction counts.

Spec conformance

• Topic Manager Pattern – Validates outputs via identifyAdmissibleOutputs() method
• Lookup Service Pattern – Stores/queries via outputAdmittedByTopic(), outputSpent(), lookup() methods
• PushDrop (BRC-48) – Standard output decoding format
• GASP – Graph Aware Sync Protocol for multi-node overlay synchronization (disable for simple local deployments)

Integration with ts-stack

• Implements topic managers and lookup services from @bsv/sdk Transaction and PushDrop utilities
• Coordinates with WalletAdvertiser for overlay service advertisement
• Uses @bsv/overlay-express server configuration and routing
• Connects wallet operations to blockchain via Arcade, Arc, Chaintracks, and Wallet Storage
• Services registered in src/index.ts: tm_protomap, ls_protomap, tm_certmap, ls_certmap, tm_uhrp, ls_uhrp, etc.

Common pitfalls

• Topic manager IDs must start with tm_, lookup service IDs with ls_
• Invalid outputs must be silently skipped in topic managers (don't throw errors)
• Lookup services use factory pattern returning instances from MongoDB connection
• GASP sync may cause conflicts if disabled on some nodes; keep consistent across deployment
• Admin API requires ADMIN_TOKEN in Authorization header; unauthenticated calls rejected
• Database indexes critical for performance with large transaction volumes
• Do not run BASM without a production chain tracker. CHAINTRACKS_URL or USE_ARCADE_CHAINTRACKS=true should be configured so anchors use canonical block headers and reorg handling.
• Keep THROW_ON_BROADCAST_FAIL=true unless the deployment intentionally accepts local overlay admission while broadcast providers are unavailable.
• Arcade-mounted Chaintracks uses /chaintracks/v2; a standalone go-chaintracks deployment may use a different prefix, so set CHAINTRACKS_API_PREFIX explicitly when needed.

Source

• GitHub
• npm package