Senior Data Engineer: Real-Time Market Data Pipelines

Hermeneutic Investments Ltd.; Hong Kong; Sur place; HKD 700 000 - 1 200 000; Plein temps

Résumé du poste

Hermeneutic Investments Ltd. in Hong Kong is seeking a Senior Data Engineer to architect, implement, and operate the end-to-end data infrastructure for our research and trading operations.

You will design scalable pipelines handling market data, news feeds, and internal sources, balancing reliability, cost, and performance. The role requires hands-on expertise across SQL, Python/Scala, ETL tooling, and modern data platforms.

Qualifications

• Strong problem-solving and analytical thinking.
• Excellent cross-functional communication.
• Proficient in building robust data quality checks.
• Experience identifying anomalies in ingested data.
• Advanced SQL and query optimization.
• Proficiency in Python or Java/Scala.
• Experience building ETL pipelines with Airflow, dbt, or scripts.
• Understanding of dimensional modeling and star/snowflake schemas.
• Experience with Snowflake, Redshift, BigQuery, Synapse, etc.
• Knowledge of Apache Spark, Kafka, Flink.
• Data security and privacy standards knowledge.

Responsabilités

• Design, build, and optimize scalable pipelines for ingesting, transforming, and integrating large-volume datasets (market data, news feeds and various unstructured data sources)
• Ensure data quality, consistency, and real-time monitoring using tools like DBT, 3rd party libraries that can facilitate data validation processes
• Develop processes to normalize and organize our data warehouse for use across different departments
• Apply advanced data management practices to ensure the scalability, availability, and efficiency of data storage
• Ensure the infrastructure supports trading and research needs while maintaining data integrity, security, and performance at scale
• Collaborate with research and analytics teams to understand their data needs and build frameworks that empower data exploration, analysis, and model development. Create tools for overlaying data from multiple sources
• Ensure that data storage, processing, and management are done in a cost-effective manner, optimizing both hardware and software resources. Implement solutions that balance high performance with cost control
• Stay ahead of the curve by continuously evaluating and adopting the most suitable technologies for the organization's data engineering needs. Ensure that company's systems align with the latest best practices in data management

Connaissances

SQL

Python/Java/Scala

ETL pipelines

Airflow/dbt

Data modeling

Snowflake/Redshift/BigQuery

Apache Spark

Kafka/Flink

Data security

Cross-functional collaboration

Problem solving

Formation

CS/Engineering/Math degree

Outils

Docker

Kubernetes

Snowflake

BigQuery

Redshift

Apache Atlas

Amundsen

Description du poste

Hermeneutic Investments Ltd. in Hong Kong is seeking a Senior Data Engineer to architect, implement, and operate the end-to-end data infrastructure for our research and trading operations.

You will design scalable pipelines handling market data, news feeds, and internal sources, balancing reliability, cost, and performance. The role requires hands-on expertise across SQL, Python/Scala, ETL tooling, and modern data platforms.