Senior Data Platform Engineer - Scalable Data Infrastructure

Selby Jennings; Chicago (IL); Hybrid; USD 150,000 - 210,000; Full time

Job summary

Selby Jennings partners with a technology-driven quantitative trading and financial markets firm to recruit a Senior Data Platform Engineer. You will help design, build, and evolve large-scale data infrastructure used across trading, research, and analytics.

The role is based in Chicago with a hybrid schedule and emphasizes data engineering, distributed systems, and platform development to support high-volume data workloads and proactive platform improvements.

Qualifications

• 5+ years of experience in Data Engineering, Data Platform Engineering, or related fields.
• Strong hands‑on development experience with Python and SQL.
• Experience building and maintaining large-scale data pipelines and distributed data systems.
• Exposure to technologies such as Kafka, Spark, Hadoop/HDFS, Flink, Iceberg, Databricks, or similar platforms.
• Solid understanding of Linux/Unix environments and production troubleshooting.
• Experience working with cloud object storage platforms such as S3 or equivalent.
• Ability to read, understand, and collaborate on application code, with Java experience considered a strong plus.
• Demonstrated ownership mindset and ability to operate in fast‑paced, highly technical environments.

Responsibilities

• Design, build, and maintain large-scale data platforms and storage systems supporting real-time and batch workloads.
• Develop and enhance data ingestion and processing pipelines using technologies such as Python, Java, Spark, Kafka, and related Big Data frameworks.
• Partner with software engineers, infrastructure teams, and data consumers to improve data accessibility, reliability, and performance.
• Support the evolution of modern data lake and table architectures, enabling scalable access to large datasets across the organization.
• Troubleshoot complex production issues, perform root-cause analysis, and drive long-term platform improvements.
• Contribute to data modeling, storage optimization, and capacity planning initiatives.
• Serve as a subject matter expert on distributed data systems and platform best practices.

Skills

Python

SQL

Distributed systems

Big data

Troubleshooting

Linux/Unix

Cloud storage (S3)

Tools

Kafka

Spark

Hadoop/HDFS

Flink

Iceberg

Databricks

Kubernetes

Docker

Job description

Selby Jennings partners with a technology-driven quantitative trading and financial markets firm to recruit a Senior Data Platform Engineer. You will help design, build, and evolve large-scale data infrastructure used across trading, research, and analytics.

The role is based in Chicago with a hybrid schedule and emphasizes data engineering, distributed systems, and platform development to support high-volume data workloads and proactive platform improvements.