Senior Data Engineer, Unified Data Platform

P2P; Singapore; On-site; SGD 90,000 - 130,000; Full time

Job summary

DRW is a diversified trading firm with over three decades of experience, operating globally and headquartered in Chicago. We seek a Senior Data Engineer to bring vendor datasets into our data platform, govern centralized pipelines, and support rapid data product development across Traders, Quant Researchers, and Back-Office teams.

You will own data ingestion pipelines, collaborate with Data Strategists, and contribute to data governance and quality control while supporting on-call rotations to

Qualifications

• 5+ years of experience with modern data technologies and/or building data-first products.
• Excellent written and verbal communication skills.
• Proven ability to work in a collaborative, agile, and fast-paced environment, prioritizing multiple tasks and projects, and efficiently handle the demands of a trading environment.
• Proven ability to deliver rapid results within processes that span multiple stakeholders.
• Strong technical problem-solving skills.
• Extensive familiarity with SQL and Java or Python, with a proven ability to develop and deliver maintainable data transformations for production data pipelines.
• Experience leveraging data modeling techniques and ability to articulate the trade-offs of different approaches.
• Experience with one or more data processing technologies (e.g. Flink, Spark, Polars, Dask, etc.).
• Experience with multiple data storage technologies (e.g. S3, RDBMS, NoSQL, Delta/Iceberg, Cassandra, Clickhouse, Kafka, etc.) and knowledge of their associated trade-offs.
• Experience with multiple data formats and serialization systems (e.g. Arrow, Parquet, Protobuf/gRPC, Avro, Thrift, JSON, etc.).
• Experience managing data pipeline orchestration systems (e.g. Kubernetes, Argo Workflows, Airflow, Prefect, Dagster, etc.).
• Proven experience in managing the operational aspects of large data pipelines such as backfilling datasets, rerunning batch jobs, and handling dead-letter queues.
• Prior experience triaging data quality control processes, correcting data gaps and inaccuracies.

Responsibilities

• Model, build, and manage data products atop DRW’s Unified Data Platform.
• Determine appropriate data sources with Data Strategists and onboard/manage new data sources for trading, research, and back-office purposes.
• Contribute to data governance processes including discovery, cost-sharing, usage tracking, access controls, and quality control of datasets.
• Continually monitor data ingestion pipelines and data quality to ensure stability, reliability, and quality of the data; contribute to monitoring tools and processes.
• Own the technical aspects of vendor ingestion pipelines, coordinating with vendor relationship managers on changes, performing routine data operations without breaking internal users, and contributing to on-call rotation.
• Rapidly respond to user requests, identifying platform gaps and self-service opportunities that improve user experience.

Skills

Data engineering experience

Communication skills

Agile collaboration

Problem solving

Tools

SQL

Java

Python

Flink

Spark

Polars

Dask

Kubernetes

Airflow

Prefect

Dagster

Job description

DRW is a diversified trading firm with over three decades of experience, operating globally and headquartered in Chicago. We seek a Senior Data Engineer to bring vendor datasets into our data platform, govern centralized pipelines, and support rapid data product development across Traders, Quant Researchers, and Back-Office teams.

You will own data ingestion pipelines, collaborate with Data Strategists, and contribute to data governance and quality control while supporting on-call rotations to