Senior Data Engineer: Real-Time Pipelines & Kafka

Fintal Partners; Chicago (IL); On-site; USD 120,000 - 160,000; Full time

Job summary

Fintal Partners in Chicago is seeking an experienced Data Engineer to design, build and roll out the Big Data stack, and to create, deploy and monitor data pipelines using Java, Python, Spark and Flink.

You will partner with development teams on data modelling, ingestion and capacity planning, ensuring data accuracy and availability while advising users and developers on data questions. This role requires 5+ years in data engineering and strong Kafka/Spark/Flink experience.

Qualifications

• 5+ years in established data engineering
• 3+ years Kafka streaming and/or running Kafka clusters
• 2+ years building apps or pipelines on Big Data backends (S3, HDFS, Databricks, Iceberg)
• Hands-on with Spark or Flink
• Strong development in Java, Python and SQL
• Familiarity with Python-based data science toolkits
• Practical Kubernetes and Docker experience
• Monitoring tooling — Prometheus/Grafana, Alert Manager, Alerta, OpsGenie
• Solid statistical analysis ability
• Proven troubleshooting and root-cause analysis
• Unix scripting (bash, Python or similar)

Responsibilities

• Design, build and roll out the Big Data stack (Kafka, Hadoop, Dremio).
• Create, deploy and monitor data pipelines using Java, Python, Spark and Flink.
• Partner with dev teams on data modelling, ingestion and capacity planning.
• Work with users to keep data accurate and available.
• Serve as the in-house Big Data expert, advising users and developers.

Skills

Big Data

Java

Python

SQL

Data modelling

Troubleshooting

Unix scripting

Statistical analysis

Root-cause analysis

Team collaboration

Tools

Kafka

Hadoop

Dremio

Spark

Flink

Kubernetes

Docker

Prometheus

Grafana

Alertmanager

Alerta

OpsGenie

Shell/Bash

Job description

Fintal Partners in Chicago is seeking an experienced Data Engineer to design, build and roll out the Big Data stack, and to create, deploy and monitor data pipelines using Java, Python, Spark and Flink.

You will partner with development teams on data modelling, ingestion and capacity planning, ensuring data accuracy and availability while advising users and developers on data questions. This role requires 5+ years in data engineering and strong Kafka/Spark/Flink experience.