Reading the Change Data Feed from a Delta Table

Reading the CDF data from a table with change data is easy.

Reading CDF Log

Python Rust

import polars
from deltalake import DeltaTable

dt = DeltaTable("../rust/tests/data/cdf-table")
table = dt.load_cdf(starting_version=0, ending_version=4).read_all()
pt = polars.from_arrow(table)
pt.group_by("_commit_version").len().sort("len", descending=True)

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {

    let table = deltalake::open_table("../rust/tests/data/cdf-table").await?;
    let ops = DeltaOps(table);
    let cdf = ops.load_cdf()
        .with_starting_version(0)
        .with_ending_version(4)
        .build()
        .await?;

    arrow_cast::pretty::print_batches(&cdf)?;

    Ok(())
}

The output can then be used in various execution engines. The python example shows how one might consume the cdf feed inside polars.