Data Reliability Engineer - Real-Time Pipelines & APIs

Selby Jennings; Chicago (IL); On-site; USD 110,000 - 160,000; Full time

Job summary

Selby Jennings is seeking a Data Reliability Engineer to join a high-performance data engineering team supporting trading and research. The role blends data engineering, platform reliability, and operational support to ensure datasets, pipelines, and APIs remain accurate and production-ready.

You will act as a primary data contact, monitor pipelines, onboard new datasets, and develop scalable ETL/ELT processes across real-time and analytical workloads.

Qualifications

• 3+ years of experience in Data Engineering, Data Reliability, Site Reliability or Data Operations.
• Strong Python development experience with Pandas, PyArrow, and Spark.
• Experience building and supporting large-scale data pipelines and APIs.
• Solid data modeling, data quality, and data architecture knowledge.
• Experience with cloud data platforms and lakehouse technologies such as Databricks, Delta Lake, or AWS.
• Exposure to financial market data or capital markets environments is a plus.
• Proven ability to troubleshoot production issues and drive operational improvements.
• Strong communication skills in fast-paced, business-critical environments.

Responsibilities

• Act as primary point of contact for data-related issues across trading, research, and business teams.
• Investigate and resolve data quality, availability, and freshness issues.
• Monitor data pipelines, workflows, and real-time feeds; respond to incidents.
• Coordinate with external data providers to resolve feed issues and discrepancies.
• Communicate incident status and resolutions to stakeholders.
• Integrate and onboard new datasets with schema mapping, transformation, validation, and history backfills.
• Develop scalable ETL/ELT pipelines for real-time and analytical workloads.
• Build data quality frameworks, monitoring, and automated validation processes.
• Design and maintain normalized data models for complex financial datasets.
• Develop and support APIs delivering data to trading, research, and analytics platforms.
• Partner with engineering, quantitative, and business teams to deliver reliable data solutions.
• Enhance observability, monitoring, and tooling to improve reliability and performance.

Skills

Data Engineering

Python

Pandas

PyArrow

Spark

Tools

Databricks

Delta Lake

AWS

Job description

Selby Jennings is seeking a Data Reliability Engineer to join a high-performance data engineering team supporting trading and research. The role blends data engineering, platform reliability, and operational support to ensure datasets, pipelines, and APIs remain accurate and production-ready.

You will act as a primary data contact, monitor pipelines, onboard new datasets, and develop scalable ETL/ELT processes across real-time and analytical workloads.