Skip to main content
Starting with Ledger v2.3, you can export transaction logs in real-time to external systems using the Logs Exporter & Pipelines feature. This enables streaming ledger logs to external sinks for analytics, observability, or integration purposes.

​
Supported Exporters

The following export drivers are available:
DriverDescription
clickhouseExport to ClickHouse database
elasticsearchExport to Elasticsearch
httpExport to any HTTP endpoint
stdoutOutput to standard output (useful for debugging)

​
Architecture

The export system consists of two main components:

Exporters

Define where and how to send data. An exporter contains the connection configuration and driver type for the destination system.

Pipelines

Define which ledger sends data to which exporter. Each pipeline connects one ledger to one exporter, enabling independent data flows.

​
Creating an Exporter

An exporter defines the destination and configuration for exported data.

​
ClickHouse Example

curl -X POST /api/ledger/v2/_/exporters \
  -H "Content-Type: application/json" \
  -d '{
    "driver": "clickhouse",
    "config": {
      "dsn": "clickhouse://default:default@clickhouse:9000/ledger"
    }
  }'
Response: 
{
  "data": {
    "id": "3fec8cd5-0bee-4cc5-8578-a71d97e39b58",
    "createdAt": "2025-10-17T09:47:59.077604Z",
    "driver": "clickhouse",
    "config": {
      "dsn": "clickhouse://default:default@clickhouse:9000/ledger"
    }
  }
}

​
Elasticsearch Example

curl -X POST /api/ledger/v2/_/exporters \
  -H "Content-Type: application/json" \
  -d '{
    "driver": "elasticsearch",
    "config": {
      "url": "http://elasticsearch:9200",
      "index": "ledger-logs"
    }
  }'

​
HTTP Example

curl -X POST /api/ledger/v2/_/exporters \
  -H "Content-Type: application/json" \
  -d '{
    "driver": "http",
    "config": {
      "url": "https://your-webhook-endpoint.com/ledger-events"
    }
  }'

​
Creating a Pipeline

A pipeline connects a ledger to an exporter. Once created, all new transaction logs from that ledger are exported through the specified exporter. 
curl -X POST /api/ledger/v2/{ledger}/pipelines \
  -H "Content-Type: application/json" \
  -d '{
    "exporterID": "3fec8cd5-0bee-4cc5-8578-a71d97e39b58"
  }'
Response: 
{
  "data": {
    "id": "6ddb1f75-ec41-469a-a8d2-82b775e3512c",
    "ledger": "ledger-002",
    "exporterID": "3fec8cd5-0bee-4cc5-8578-a71d97e39b58",
    "createdAt": "2025-10-17T09:52:25.648245Z",
    "enabled": true
  }
}

​
Managing Pipelines

​
List Pipelines

curl -X GET /api/ledger/v2/{ledger}/pipelines

​
Get Pipeline State

curl -X GET /api/ledger/v2/{ledger}/pipelines/{pipelineId}

​
Start/Stop Pipeline

# Start pipeline
curl -X POST /api/ledger/v2/{ledger}/pipelines/{pipelineId}/start

# Stop pipeline
curl -X POST /api/ledger/v2/{ledger}/pipelines/{pipelineId}/stop

​
Reset Pipeline

Reset the pipeline cursor to re-export logs from the beginning: 
curl -X POST /api/ledger/v2/{ledger}/pipelines/{pipelineId}/reset

​
Delete Pipeline

curl -X DELETE /api/ledger/v2/{ledger}/pipelines/{pipelineId}

​
Managing Exporters

​
List Exporters

curl -X GET /api/ledger/v2/_/exporters

​
Get Exporter State

curl -X GET /api/ledger/v2/_/exporters/{exporterId}

​
Delete Exporter

curl -X DELETE /api/ledger/v2/_/exporters/{exporterId}
You cannot delete an exporter that has active pipelines connected to it. Delete or reassign the pipelines first.

​
Throughput and Rate Limiting

You can attach multiple ledgers (pipelines) to the same exporter. This design allows you to control throughput and parallelism while avoiding rate limits on external systems.

​
Why use multiple pipelines?

  • Avoid rate limits: Some external sinks (e.g., Elasticsearch, HTTP APIs) may rate-limit incoming writes
  • Better concurrency: Spread load across multiple pipelines instead of one handling massive volume
  • Isolation: Each ledger maintains its own export pipeline, ensuring independent data flows
  • Scalability: Simplify scaling strategies by partitioning exports by ledger

​
Example: Multi-ledger setup

Connect multiple ledgers to the same ClickHouse exporter: 
# Create pipelines for each ledger
curl -X POST /api/ledger/v2/ledger-001/pipelines \
  -d '{"exporterID": "<exporter-id>"}'

curl -X POST /api/ledger/v2/ledger-002/pipelines \
  -d '{"exporterID": "<exporter-id>"}'

curl -X POST /api/ledger/v2/ledger-003/pipelines \
  -d '{"exporterID": "<exporter-id>"}'
This approach helps:
  • Prevent overload of a single export stream
  • Simplify scaling strategies
  • Centralize monitoring of export targets

​
Summary

ComponentPurposeExample
ExporterDefines the destination (ClickHouse, HTTP, etc.)clickhouse://...
PipelineConnects a ledger to an exporterledger-002 → clickhouse exporter
Multiple Pipelines per ExporterImproves throughput, avoids rate limitsledger-001, ledger-002, ledger-003 → same exporter
For webhook-style event publishing on Formance Cloud, see also Webhooks and Events Publishers for HTTP and Kafka publishing options available in earlier versions.