Remote Senior Market Data Engineer

BHFT; United Arab Emirates; Remote; AED 240,000 - 360,000; Full time

Benefits offered by this job

Remote work

Flexible schedule

Health insurance

Sports activities

Professional training

Job summary

BHFT is seeking a data engineering specialist to design, build and maintain robust Historical Data Capture and Storage systems for cross-exchange data. You will ensure data integrity, implement ETL pipelines, and enable fast access for backtesting and research in a remote, globally distributed setup.

You will collaborate with trading and research teams, optimize storage and query performance, and drive cost-efficient, compliant data solutions across large-scale time series datasets.

Qualifications

• Commercial experience of financial instruments and markets (equities futures options forex etc.) particularly understanding how historical data is used for algorithmic trading.
• Familiarity with market data formats (e.g. MDP ITCH FIX SWIFT proprietary exchange APIs) and market data providers.
• Strong programming skills in Python (Go/Rust is a nice to have)
• Familiarity with ETL (Extract Transform Load) processes (or other data pipeline architecture) and tools to clean normalize and validate large datasets.
• Commercial experience in building and maintaining large-scale time series or historical market data in the financial services industry.
• Strong SQL proficiency: aggregations joins subqueries window functions (first last candle histogram) indexes query planning and optimization.
• Strong problem-solving skills and attention to detail particularly in ensuring data quality and reliability.
• Bachelors degree in Computer Science Engineering or related field.

Responsibilities

• Design, develop and maintain systems for the acquisition, storage and retrieval of historical market data from multiple financial exchanges, brokers and market data vendors.
• Ensure the integrity and accuracy of historical market data including implementing data validation, cleansing and normalization processes.
• Build and optimize data storage solutions ensuring they are scalable, high-performance and capable of managing large volumes of time-series data.
• Develop systems for data versioning and reconciliation to handle changes in exchange formats or corrections to past data.
• Implement robust integrations with market data providers, exchanges and data sources to continuously collect and store historical data.
• Build internal tools to provide easy access to historical data for research and analysis, ensuring performance, ease of use and data integrity.
• Work closely with quantitative researchers and traders to understand data requirements and optimize systems for data retrieval and analysis for backtesting and strategy development.
• Develop scalable solutions to handle growing volumes of historical market data, including efficient queries and data retrieval for research and backtesting needs.
• Optimize data storage costs balancing cost-efficiency with performance and ensuring large datasets are managed effectively.
• Ensure historical market data systems comply with regulatory requirements and assist in data retention, integrity and reporting audits.

Skills

Python

SQL

ETL pipelines

Data cleansing

Time series data

Data quality

Large-scale data

Problem solving

Attention to detail

Education

Bachelors degree in Computer Science / Engineering / related field

Tools

Docker

Airflow

SLURM

Hadoop

Kafka

Spark

Job description

BHFT is seeking a data engineering specialist to design, build and maintain robust Historical Data Capture and Storage systems for cross-exchange data. You will ensure data integrity, implement ETL pipelines, and enable fast access for backtesting and research in a remote, globally distributed setup.

You will collaborate with trading and research teams, optimize storage and query performance, and drive cost-efficient, compliant data solutions across large-scale time series datasets.