Index Product Search From Change Data Capture

Product search is rebuilt every night by a job that reads all 6.2 million products from the catalog database and writes a fresh Elasticsearch index. Price and stock changes made during the day don't reach search until the next run, and merchants filed 1,900 tickets last quarter about discontinued products that were still searchable.

Debezium reads the catalog database's write-ahead log through a logical replication slot on the primary and publishes row changes for the products, product_variants, and categories tables to Kafka, one topic per table.

An indexer service consumes the three topics, loads each affected product with its variants and category path from the catalog read replica, and upserts the resulting document into the products index.

Each change is written to Elasticsearch as its own index request, so a document is searchable as soon as the request returns.

Each topic is keyed by the table's primary key, and the indexer runs 12 consumers. A delete event removes the product's document from the index, and any other event upserts it.

The connector adds one replication connection to the primary, and staging showed under 2% added CPU. If the connector stops, the replication slot keeps its position, and the connector resumes from where it left off with no lost changes.

Teams that change the catalog schema will post in #search-indexing before their migration ships, so the Search team can update the indexer.

We'll alert when the Debezium connector or the indexer has been down for more than 5 minutes.

The CDC indexer runs alongside the nightly job for two weeks, writing to a shadow index that we compare against production. After that, search reads from the CDC index, and the nightly reindex job and its Airflow DAG are deleted.

Inventory freshness is out of scope for this project.

Catalog changes are searchable within 60 seconds at p99, and tickets about discontinued products drop below 200 a quarter.