Principal Data Engineer

Fintal Partners; Chicago (IL); On-site; USD 180,000 - 240,000; Full time

Job summary

Fintal Partners in Chicago is seeking an experienced Principal Data Engineer to join our fast-moving team. You will design, build and roll out our Big Data platform, leveraging Kafka, Hadoop and Dremio to power trading insights.

You will develop and monitor data pipelines with Java, Python, Spark and Flink, partner with engineering on data modeling and capacity planning, and act as the go-to expert for data questions across users and developers.

Qualifications

• 5+ years in an established data engineering environment.
• 3+ years building Kafka streaming applications and/or managing Kafka clusters.
• 2+ years developing pipelines or applications on Big Data backends (S3, HDFS, Databricks, Iceberg).
• Experience working in an on-prem Linux environment is highly preferred.
• Experience with Apache Spark, Apache Flink or comparable tools.
• Strong Java, Python and SQL skills.
• Familiarity with Python-based data science tools.
• Practical experience with Kubernetes and Docker.
• Exposure to monitoring tools such as Prometheus/Grafana, Alert Manager, Alerta and OpsGenie.
• Solid statistical analysis skills.
• Proven troubleshooting and root-cause analysis ability.

Responsibilities

• Design, build and roll out our Big Data platform (Kafka, Hadoop, Dremio, etc.)
• Develop, deploy and monitor data processing pipelines (Java, Python, Spark, Flink)
• Partner with engineering teams on data modeling, ingestion and capacity planning
• Work with users to keep data accurate and accessible
• Serve as the go-to Big Data expert, advising users and developers on data questions

Skills

Kafka

Hadoop

Spark

Flink

Python

Java

SQL

Data modeling

Docker

Kubernetes

Monitoring tools

Statistical analysis

Troubleshooting

Tools

Databricks

Iceberg

S3

HDFS

Prometheus

Grafana

Docker

Kubernetes

Job description

We're looking for an experienced, driven Principal Data Engineer to join our Chicago office. You'll do well here if you enjoy a fast-moving, innovative environment that keeps the firm at the leading edge of global financial markets.

What You'll Do

• Design, build and roll out our Big Data platform (Kafka, Hadoop, Dremio, etc.)
• Develop, deploy and monitor data processing pipelines (Java, Python, Spark, Flink)
• Partner with engineering teams on data modeling, ingestion and capacity planning
• Work with users to keep data accurate and accessible
• Serve as the go-to Big Data expert, advising users and developers on data questions

What You'll Bring

• 5+ years in an established data engineering environment
• 3+ years building Kafka streaming applications and/or managing Kafka clusters
• 2+ years developing pipelines or applications on Big Data backends (S3, HDFS, Databricks, Iceberg, etc.)
• Experience working in an "on-prem" / Linux environment is highly preferred
• Experience with Apache Spark, Apache Flink or comparable tools
• Strong Java, Python and SQL skills
• Familiarity with common data science toolkits, particularly Python-based
• Practical experience with Kubernetes and Docker
• Exposure to monitoring tools such as Prometheus/Grafana, Alert Manager, Alerta and OpsGenie
• Solid statistical analysis skills
• Proven troubleshooting and root-cause analysis ability

About the Firm

Our client is a research-led proprietary trading firm where quant modeling, machine learning and engineering drive how markets are traded. A steady liquidity provider for over three decades, the firm trades across global venues, delivering strong value and risk outcomes for investors. It runs on its own capital and in-house technology, building proprietary systems and algorithms. Researchers, traders and engineers work as one team, using fast experimentation, advanced infrastructure and real-time feedback to turn ideas into execution and execution into an edge.