Senior Data Engineer

CMC Markets; Województwo mazowieckie; On-site; PLN 210,000 - 320,000; Full time

Job summary

CMC Markets is seeking a Senior Data Engineer to design and operate a high-performance market data platform for quantitative research and live trading systems. You will build scalable data pipelines and deliver robust, production-ready datasets for research and real-time analytics.

The role requires expertise in Python, SQL, time-series data, and modern storage formats, with experience in distributed processing and streaming technologies.

Qualifications

• Proven experience designing and operating scalable market data platforms.
• Strong background in building high-volume ETL/ELT pipelines for real-time data.
• Hands-on experience with time-series data challenges and data quality controls.
• Familiarity with distributed processing frameworks and streaming technologies.

Responsibilities

• Build scalable ETL/ELT pipelines for batch and streaming market data.
• Develop production-grade Python and SQL solutions.
• Ingest, normalise and validate market data including trades and quotes.
• Implement automated data quality, reconciliation and monitoring frameworks.
• Optimise data storage using Parquet and modern data lake tech.
• Partner with researchers, traders and engineers to deliver trusted datasets.
• Experience with Docker, Linux, Git and CI/CD.

Skills

Python

SQL

ETL/ELT pipelines

Data quality

Time-series data

Distributed processing

Kafka

PostgreSQL

ClickHouse

Snowflake

Databricks

Docker

Linux

CI/CD

Tools

Spark

Kafka

PostgreSQL

ClickHouse

Snowflake

Databricks

kdb+

Parquet

Arrow

Dask

Ray

Git

Job description

We are looking for a Senior Data Engineer to build and operate a high-performance market data platform supporting quantitative research and production trading systems. You'll design scalable data pipelines, process billions of market data records, and deliver reliable, production-quality datasets for research, analytics and real-time applications.

Key Responsibilities

• Build scalable ETL/ELT pipelines for batch and streaming market data.
• Develop production-grade Python and SQL solutions.
• Ingest, normalise and validate market data, including trades, ticks, Level 1 & 2 quotes, order books, reference data and corporate actions.
• Implement automated data quality, reconciliation and monitoring frameworks.
• Optimise data storage using Parquet, Arrow and modern data lake technologies.
• Partner with quantitative researchers, traders and engineering teams to deliver trusted datasets.
• Strong Python and SQL.
• Expertise with Parquet and modern data storage formats.
• Deep understanding of high-frequency market data, including tick data, order books and market data normalisation.
• Experience handling time-series challenges such as timestamp precision, sequence gaps, duplicate and out-of-order events.
• Experience with distributed processing (Spark, Polars, Dask, Ray or similar).
• Experience with Kafka or similar streaming technologies.
• Familiarity with PostgreSQL, ClickHouse, Snowflake, Databricks, kdb+ or similar.
• Docker, Linux, Git and CI/CD experience.

Highly Desirable

• Familiarity with OneTick, PostgreSQL, ClickHouse, Snowflake, Databricks, kdb+ or similar.
• Order book reconstruction
• Airflow, Dagster, Prefect or MLflow
• Experience supporting quantitative research or trading systems

What We're Looking For

The successful candidate will have a strong background in market data engineering with proven experience building reliable, scalable data platforms.

Our non-negotiable requirements are:

• Expert Python
• Strong SQL
• Automated data quality controls
• Deep tick and market data expertise

Experience in MLOps is advantageous but not essential. We welcome candidates with strong experience in either market data engineering or MLOps, provided they have the skills to build and operate production-grade data platforms.