Senior Data Platform Engineer

Selby Jennings; Chicago (IL); Hybrid; USD 150,000 - 210,000; Full time

Job summary

Selby Jennings partners with a technology-driven quantitative trading and financial markets firm to recruit a Senior Data Platform Engineer. You will help design, build, and evolve large-scale data infrastructure used across trading, research, and analytics.

The role is based in Chicago with a hybrid schedule and emphasizes data engineering, distributed systems, and platform development to support high-volume data workloads and proactive platform improvements.

Qualifications

• 5+ years of experience in Data Engineering, Data Platform Engineering, or related fields.
• Strong hands‑on development experience with Python and SQL.
• Experience building and maintaining large-scale data pipelines and distributed data systems.
• Exposure to technologies such as Kafka, Spark, Hadoop/HDFS, Flink, Iceberg, Databricks, or similar platforms.
• Solid understanding of Linux/Unix environments and production troubleshooting.
• Experience working with cloud object storage platforms such as S3 or equivalent.
• Ability to read, understand, and collaborate on application code, with Java experience considered a strong plus.
• Demonstrated ownership mindset and ability to operate in fast‑paced, highly technical environments.

Responsibilities

• Design, build, and maintain large-scale data platforms and storage systems supporting real-time and batch workloads.
• Develop and enhance data ingestion and processing pipelines using technologies such as Python, Java, Spark, Kafka, and related Big Data frameworks.
• Partner with software engineers, infrastructure teams, and data consumers to improve data accessibility, reliability, and performance.
• Support the evolution of modern data lake and table architectures, enabling scalable access to large datasets across the organization.
• Troubleshoot complex production issues, perform root-cause analysis, and drive long-term platform improvements.
• Contribute to data modeling, storage optimization, and capacity planning initiatives.
• Serve as a subject matter expert on distributed data systems and platform best practices.

Skills

Python

SQL

Distributed systems

Big data

Troubleshooting

Linux/Unix

Cloud storage (S3)

Tools

Kafka

Spark

Hadoop/HDFS

Flink

Iceberg

Databricks

Kubernetes

Docker

Job description

We are partnering with a leading technology-driven quantitative trading and financial markets firm seeking a Senior Data Platform Engineer to join a team responsible for building and evolving large-scale data infrastructure. This role sits at the intersection of data engineering, distributed systems, and platform development, supporting high-volume, mission-critical data used across trading, research, and analytics functions.

What You’ll Do

Chicago, IL (Hybrid - 4 Days Onsite)

We are partnering with a leading technology-driven quantitative trading and financial markets firm seeking a Senior Data Platform Engineer to join a team responsible for building and evolving large-scale data infrastructure. This role sits at the intersection of data engineering, distributed systems, and platform development, supporting high-volume, mission-critical data used across trading, research, and analytics functions.

• Design, build, and maintain large-scale data platforms and storage systems supporting real-time and batch workloads.
• Develop and enhance data ingestion and processing pipelines using technologies such as Python, Java, Spark, Kafka, and related Big Data frameworks.
• Partner with software engineers, infrastructure teams, and data consumers to improve data accessibility, reliability, and performance.
• Support the evolution of modern data lake and table architectures, enabling scalable access to large datasets across the organization.
• Troubleshoot complex production issues, perform root-cause analysis, and drive long-term platform improvements.
• Contribute to data modeling, storage optimization, and capacity planning initiatives.
• Serve as a subject matter expert on distributed data systems and platform best practices.

What We’re Looking For

• 5+ years of experience in Data Engineering, Data Platform Engineering, or related fields.
• Strong hands‑on development experience with Python and SQL.
• Experience building and maintaining large-scale data pipelines and distributed data systems.
• Exposure to technologies such as Kafka, Spark, Hadoop/HDFS, Flink, Iceberg, Databricks, or similar platforms.
• Solid understanding of Linux/Unix environments and production troubleshooting.
• Experience working with cloud object storage platforms such as S3 or equivalent.
• Ability to read, understand, and collaborate on application code, with Java experience considered a strong plus.
• Demonstrated ownership mindset and ability to operate in fast‑paced, highly technical environments.

Preferred Background

• Experience supporting large‑scale data platforms serving engineering, analytics, or research organizations.
• Exposure to modern lakehouse architectures and large‑volume data storage solutions.
• Experience with Kubernetes, Docker, monitoring, and observability tooling.
• Background in financial services, quantitative trading, large‑scale technology companies, or other data‑intensive environments is highly valued.

Why Consider This Opportunity?

• Join a highly technical team solving complex distributed systems and data challenges.
• Work on modern data platform initiatives involving next‑generation storage and processing technologies.
• Influence critical infrastructure used across the business.
• Collaborative environment with significant ownership and visibility.