About this role
Builds and runs real time streaming pipelines, keeping latency low and jobs recoverable at high volume.
- Strong hands on Apache Flink across DataStream API and Flink SQL.
- Event time processing, checkpointing, savepoints, and state management in production.
- High volume Kafka streaming.
- Data modelling for analytics and streaming into a lakehouse.
- Monitors pipeline health and handles recovery when jobs fail.
Requirements
Requirements
-
pache Flink
-
(DataStream, Flink SQL), Kafka, Java, Scala, Python, SQL, Docker, Kubernetes
-
Strong hands-on experience with Apache Flink, including DataStream API and Flink SQL.
-
Experience with high-volume Kafka streaming and real-time data processing.
-
Strong knowledge of event time processing, checkpointing, savepoints, and state management.
-
Experience with data modelling for analytics and streaming into a lakehouse.
-
Experience monitoring pipeline health and handling job recovery and failures.
-
Apache Iceberg or other open table formats.
-
Airflow.
-
On premises or self managed cluster experience.