Revision Tree Mechanics in CouchDB Replication #

CouchDB tracks every document’s lineage as a revision tree — a directed structure in which each revision has exactly one parent and branches form only when writes diverge before they have synchronized. For edge/IoT deployments, mobile backend engineers, and Python sync pipeline builders, treating this tree as an append-only event log is the difference between a resilient sync workflow and silent data loss. This page is the mechanics layer of CouchDB Replication Architecture & Revision Fundamentals: it shows how to configure revision retention, observe branch divergence from the _changes feed, resolve conflicting leaves in production Python, and tune compaction so the tree never grows unbounded. The branches you see here are produced by the concurrency patterns catalogued in conflict generation models, and the field-level _rev format that identifies each node is dissected in how CouchDB revision IDs are generated.

Configuration Schema & Required Parameters #

Two settings govern how much of the revision tree CouchDB keeps and how completely a replication job carries divergent branches to the target. The first is the per-database _revs_limit, which caps the depth of revision history retained per document; the second is the replication style, which decides whether every leaf or only the winner crosses the wire. Deploy the replication job itself with the standard _replicator document schema; the revision-tree-specific configuration is applied directly to the database:

{
  "database": "iot_telemetry",
  "revs_limit_endpoint": "PUT /iot_telemetry/_revs_limit",
  "revs_limit_body": 1000,
  "replicator_document": {
    "_id": "rep_edge_to_central",
    "source": "https://edge-node-01.local:5984/iot_telemetry",
    "target": "https://central-db.cluster:5984/iot_telemetry",
    "continuous": true,
    "create_target": false,
    "style": "all_docs"
  }
}

_revs_limit defaults to 1000; you set it with a bare integer body, e.g. PUT /iot_telemetry/_revs_limit with 1000. The style field belongs to the replication request, not the database — all_docs (the replicator default) copies every leaf revision so conflicts survive the trip, while main_only would ship only the winning revision and hide divergence.

Setting _revs_limit too aggressively is the classic self-inflicted wound: if a replica falls behind by more revisions than the limit retains, CouchDB prunes the common ancestor, and what should have been a fast-forward update is instead recorded as a conflict. Validate the active value with GET /iot_telemetry/_revs_limit before tuning it, and consult the Apache CouchDB replication documentation for the authoritative pruning semantics.

Streaming Detection / Monitoring Setup #

Branch divergence must be observed at the point the pipeline consumes the _changes feed, before any downstream compaction runs. Request the feed with style=all_docs so every leaf is reported, and conflicts=true so each document carries its computed _conflicts array. The minimal listener below streams changes and yields only the documents whose revision tree has actually forked:

import json

import httpx


def stream_divergent_docs(db_url: str, since: str = "now"):
    """Yield (doc_id, winning_rev, losing_revs[]) for each forked document.

    style=all_docs makes CouchDB report every leaf revision, and
    conflicts=true attaches the computed _conflicts array so the losing
    leaves ride along without a second round trip.
    """
    params = {
        "feed": "continuous",
        "since": since,
        "style": "all_docs",
        "include_docs": "true",
        "conflicts": "true",
        "heartbeat": "10000",
    }
    with httpx.stream("GET", f"{db_url}/_changes", params=params, timeout=None) as r:
        for line in r.iter_lines():
            if not line:  # heartbeat keep-alive line
                continue
            row = json.loads(line)
            doc = row.get("doc") or {}
            losers = doc.get("_conflicts")
            if losers:  # only forked docs carry this key
                yield doc["_id"], doc["_rev"], losers


if __name__ == "__main__":
    for doc_id, rev, losers in stream_divergent_docs("http://localhost:5984/iot_telemetry"):
        print(f"forked: {doc_id} winner={rev} losers={losers}")

CouchDB picks a winner among competing leaves deterministically — highest generation number first, then the lexicographically highest revision hash as a tiebreaker — but that choice only decides which revision a default read returns; it never deletes the losing leaves. Detection must therefore stay idempotent: during network flapping the same fork reappears on the feed, so the resolver has to tolerate re-processing a document whose losers were already tombstoned. The tree below shows the shape the listener is watching for.

Both generation-3 leaves share parent 2-b714; CouchDB returns 3-c20e by default because, at equal generation, the lexicographically higher hash wins (c20e > 9ef1). The losing leaf 3-9ef1 is not deleted — it persists until an application resolves the conflict. Emit a per-document metric at this point (fork detected, timestamp) so you can later measure end-to-end resolution latency.

Core Implementation #

Resolving a fork means fetching every competing leaf, applying a deterministic merge, and committing the merged winner and a tombstone for each loser in a single _bulk_docs batch. Splitting those two writes is the canonical mistake — writing a new winner alone just adds another leaf, and the document stays conflicted until each losing rev is deleted. The class below adds structured logging and bounded exponential-backoff retries on 409 Conflict, and never mutates revision history by hand: it only ever extends the tree.

import logging
import time
from typing import Any, Optional
from urllib.parse import urljoin

import requests

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
logger = logging.getLogger("revision-tree")


class RevisionTreeResolver:
    """Resolve forked CouchDB documents by merging every leaf and tombstoning losers."""

    def __init__(self, base_url: str, db_name: str,
                 auth: Optional[tuple] = None, max_retries: int = 5):
        self.base_url = base_url.rstrip("/")
        self.db_url = urljoin(self.base_url + "/", f"{db_name}/")
        self.max_retries = max_retries
        self.session = requests.Session()
        if auth:
            self.session.auth = auth

    def _get_doc(self, doc_id: str, params: dict) -> dict:
        resp = self.session.get(urljoin(self.db_url, doc_id), params=params, timeout=30)
        resp.raise_for_status()
        return resp.json()

    def _bulk_docs(self, docs: list) -> Any:
        """POST a batch, retrying on 409 with capped exponential backoff."""
        for attempt in range(1, self.max_retries + 1):
            resp = self.session.post(
                urljoin(self.db_url, "_bulk_docs"), json={"docs": docs}, timeout=30
            )
            if resp.status_code != 409:
                resp.raise_for_status()
                return resp.json()
            backoff = min(2 ** attempt, 30)
            logger.warning("409 on _bulk_docs, retrying in %ss", backoff)
            time.sleep(backoff)
        raise RuntimeError("exhausted retries committing merged revision")

    def resolve(self, doc_id: str) -> dict:
        """Fetch a document, merge all leaves deterministically, and commit."""
        # 1. Read the winner plus its computed conflict metadata.
        doc = self._get_doc(doc_id, {"conflicts": "true", "revs_info": "true"})
        losing_revs = doc.get("_conflicts")
        if not losing_revs:
            return {"status": "clean", "doc_id": doc_id}

        logger.info("resolving %s (%d losing leaves)", doc_id, len(losing_revs))

        # 2. Retrieve every losing leaf by explicit rev.
        losers = [self._get_doc(doc_id, {"rev": rev}) for rev in losing_revs]

        # 3. Deterministic field-union merge: prefer non-null values, then the
        #    higher revision hash as a stable tiebreak so the result converges.
        merged = doc.copy()
        for loser in losers:
            for key, value in loser.items():
                if key.startswith("_") or value is None:
                    continue
                if merged.get(key) is None or loser["_rev"] > merged["_rev"]:
                    merged[key] = value

        # 4. Keep _id and the winning _rev so the write extends the winning branch;
        #    drop the read-only computed fields before writing.
        merged.pop("_conflicts", None)
        merged.pop("_revs_info", None)

        # 5. Commit the merged winner AND a tombstone for every loser in one batch.
        #    Writing the winner alone would NOT clear the conflict.
        batch = [merged] + [
            {"_id": doc_id, "_rev": rev, "_deleted": True} for rev in losing_revs
        ]
        result = self._bulk_docs(batch)
        logger.info("resolved %s; %d loser(s) tombstoned", doc_id, len(losing_revs))
        return {"status": "resolved", "doc_id": doc_id, "bulk_result": result}


if __name__ == "__main__":
    resolver = RevisionTreeResolver("http://localhost:5984", "iot_telemetry")
    print(resolver.resolve("telemetry.sensor-42"))

On a 409, re-read the current winner before rebuilding the batch so you never write against a stale generation. When draining a large _conflicts backlog, bound the number of concurrent revision fetches so I/O-bound reads do not overwhelm a resource-constrained edge cluster. The merge shown here is a field-union; the decision of which algorithm to bind to each document class is covered next and, in depth, in algorithm selection for merge.

Strategy Variants & Trade-offs #

Once the losing leaves are in hand, the merge function decides how the branches collapse into a single winner. Three paradigms dominate production pipelines, each optimal for a different workload and each interacting differently with the revision tree:

Last-write-wins compares generation sequences or application timestamps and keeps a single leaf, so it collapses the tree fastest but discards a branch; its failure mode is clock skew, mitigated with NTP and hybrid logical clocks. Field-union folds non-overlapping field updates across every branch — accepting a new firmware_version from one leaf while keeping a network_config edited on another — preserving structured data at moderate CPU cost. CRDTs embed commutative, associative, idempotent merge state in every write so branches converge mathematically, at the price of payload overhead and custom schemas. Whichever you choose, the resolver must still tombstone the losing leaves, because CouchDB never prunes a live branch on its own. When a merge cannot be trusted — semantic validation fails or confidence falls below threshold — do not force a lossy write; escalate through error handling & retry logic rather than corrupting the tree.

Deployment & Orchestration #

Run the resolver as a small stateless service, one replica per replication partition. Two replicas against the same partition race to tombstone the same losers and simply double the 409 retry pressure, so scale horizontally by sharding document namespaces, not by cloning workers onto one stream. Drive every setting through the environment so one image serves every edge node:

# Container environment (one replica per partition)
COUCH_URL=https://central-db.cluster:5984
COUCH_DB=iot_telemetry
RESOLVER_MAX_RETRIES=5
COMPACT_CRON="0 3 * * *"     # nightly, low-traffic window
HEALTHCHECK_PORT=8080

Deleting the losing leaves clears a conflict, but their tombstones and the now-historical interior revisions still occupy disk until compaction runs. POST /iot_telemetry/_compact rewrites the .couch file, discarding non-leaf revision bodies beyond _revs_limit and reclaiming space from superseded revisions — but it never removes a live conflict leaf, so resolution must delete the losers first. Schedule compaction in low-traffic windows to avoid I/O contention on constrained edge devices. Expose a /healthz endpoint that confirms the worker can still reach CouchDB and that its _changes cursor is advancing; a cursor stalled longer than one heartbeat interval signals a dead listener, not a healthy idle. Pin the since checkpoint in durable storage so a restart resumes rather than replays the feed.

Troubleshooting & Common Errors #

FAQ #

Related #

• How CouchDB Revision IDs Are Generated
• Visualizing Revision Trees in CouchDB
• Conflict Generation Models
• Sync Topology Models
• Security Boundaries in Replication
• Algorithm Selection for Merge

Part of: CouchDB Replication Architecture & Revision Fundamentals