kenju's blog

About Programming and Mathematics

Resources To Read Before Developing Streaming Engine Architecture

I am developing a realtime log architecture for advertising delivery system. While researching about best practices or common solution, I found a bunch of useful resources to read through. Here is a memo to myself about those resources.

Streaming

These two essays are must to read.

Watermarks

Watermarks is the timestamp to clarify until that time log records have been processed.

Lambda Architecture

Kappa Architecture

Apache Flink

Apache Spark