Sponsor of the Day:
Jerkmate
https://jack-vanlightly.com/analyses/2024/4/24/understanding-apache-hudi-consistency-model-part-2
Understanding Apache Hudi's Consistency Model Part 2 — Jack Vanlightly
Apr 24, 2024 - In part 1 we built up an understanding of the mechanics of Hudi Copy-on-write tables, with a special regard to multi-writer scenarios, using a simplified...
model part 2apache hudijack vanlightlyunderstandingconsistency
https://jack-vanlightly.com/blog/2025/10/8/beyond-indexes-how-open-table-formats-optimize-query-performance
Beyond Indexes: How Open Table Formats Optimize Query Performance — Jack Vanlightly
Oct 8, 2025 - My career in data started as a SQL Server performance specialist, which meant I was deep into the nuances of indexes, locking and blocking, execution plan...
open table formatsquery performancejack vanlightlybeyondindexes
https://jack-vanlightly.com/blog/2025/12/10/the-three-durable-function-forms
The Three Durable Function Forms — Jack Vanlightly
Dec 10, 2025 - Durable execution engines (DEEs) talk about “workflows”, “activities”, “virtual objects”, “handlers”, and “functions”, but they’re often describing the same...
jack vanlightlythreedurablefunctionforms
https://jack-vanlightly.com/blog/category/Data
Jack Vanlightly
{ dist sys, formal verification, event streaming }
jack vanlightly
https://jack-vanlightly.com/blog/2025/6/11/coordinated-progress-part-3-coupling-synchrony-and-complexity
Coordinated Progress – Part 3 – Coupling, Synchrony and Complexity — Jack Vanlightly
Jun 11, 2025 - In part 2, we built a mental framework using a graph of nodes and edges to represent distributed work. Workflows are subgraphs coordinated via choreography or...
part 3jack vanlightlycoordinatedprogresscoupling
https://jack-vanlightly.com/blog/2025/11/19/have-your-iceberg-cubed-not-sorted-meet-qbeast-the-otree-spatial-index
Have your Iceberg Cubed, Not Sorted: Meet Qbeast, the OTree Spatial Index — Jack Vanlightly
Nov 19, 2025 - In today’s post I want to walk through a fascinating indexing technique for data lakehouses which flips the role of the index in open table formats like Apache...
jack vanlightlyicebergcubedsortedmeet
https://jack-vanlightly.com/blog/2025/10/22/a-fork-in-the-road-deciding-kafkas-diskless-future
A Fork in the Road: Deciding Kafka’s Diskless Future — Jack Vanlightly
Oct 24, 2025 - “ The Kafka community is currently seeing an unprecedented situation with three KIPs ( KIP-1150 , KIP-1176 , KIP-1183) simultaneously addressing the same...
jack vanlightlyforkroaddecidingdiskless
https://jack-vanlightly.com/
Jack Vanlightly
{ dist sys, formal verification, event streaming }
jack vanlightly
https://jack-vanlightly.com/blog/2025/12/4/the-durable-function-tree-part-1
The Durable Function Tree - Part 1 — Jack Vanlightly
Dec 10, 2025 - In my last post I wrote a bout why and where determinism is needed in durable execution (DE). In this post I'm going to explore how workflows can be formed...
tree part 1jack vanlightlydurablefunction