Senior Data Engineer - Real-Time Data Pipelines (Kafka/Spark)

IMC Trading

Chicago (IL)

On-site

USD 175,000 - 225,000

Full time

9 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

IMC Trading in Chicago seeks an experienced Data Engineer to join our growth-driven data team. You will architect and deploy large-scale data systems, build streaming and batch pipelines, and collaborate on data modeling, ingestion, and capacity planning to ensure data integrity and availability across the organization.

You will work with Java, Python, Spark, and Flink, leveraging Kubernetes and Docker for deployment while monitoring with Prometheus/Grafana.

Qualifications

5+ years experience in a mature data engineering environment.
3+ years building Kafka streaming applications and/or maintaining Kafka clusters.
2+ years building applications/pipelines with Big Data backends (S3, HDFS, Databricks, Iceberg).
Experience with Apache Spark, Apache Flink or similar tools.
Strong Java, Python, and SQL development skills.
Experience with common data-science toolkits, especially python-based.

Responsibilities

Architect, develop and deploy our Big Data environment (Kafka, Hadoop, Dremio, etc.).
Build, deploy, and monitor our data processing pipelines (Java, Python, Spark, Flink).
Collaborate with development teams on data modeling, data ingestion, and capacity planning.
Work with users to ensure data integrity and availability.
Act as a Big Data SME and consult on a variety of data-related questions from users and developers.