Senior Data Engineer - Hermeneutic Investments

Job Description

We are looking for a Senior Data Engineer to help us architect, implement, and operate the complete data infrastructure pipeline for our Research and Trading operations. This role is crucial in building a scalable, reliable, and cost-efficient system for handling vast amounts of market trading data, real-time news feeds, and various internal and external data sources. The ideal candidate will be a hands-on professional who understands the entire data lifecycle and can drive innovation while collaborating across research and engineering teams to meet their needs.

Responsibilities

• Design, build, and optimize scalable pipelines for ingesting, transforming, and integrating large datasets (market data, news feeds, unstructured data).
• Ensure data quality, consistency, and real-time monitoring using tools like DBT and other data validation libraries.
• Develop processes to normalize and organize our data warehouse for cross-departmental use.
• Apply advanced data management practices to ensure scalability, availability, and storage efficiency.
• Support trading and research needs while maintaining data integrity, security, and performance at scale.
• Collaborate with research and analytics teams to understand their data needs and build frameworks for data exploration, analysis, and modeling. Create tools for overlaying data from multiple sources.
• Implement cost-effective data storage, processing, and management solutions, balancing performance and resource use.
• Continuously evaluate and adopt suitable technologies to meet data engineering needs, aligning with best practices.

Requirements

Must Have

• Strong problem-solving and analytical skills
• Excellent communication skills for cross-functional collaboration
• Proficiency in building robust data quality checks and anomaly detection in ingested data
• Expertise in writing complex SQL queries and optimizing performance
• Proficiency in Python or Java/Scala
• Experience building and maintaining complex ETL pipelines with tools like Apache Airflow, dbt, or custom scripts
• Understanding of dimensional modeling, star/snowflake schemas, normalization/denormalization principles
• Proven experience with platforms like Snowflake, Redshift, BigQuery, or Synapse
• Expertise in Apache Spark, Kafka, Flink, or similar technologies
• Knowledge of data security and privacy standards

Good to Have

• A degree in Computer Science, Engineering, Mathematics, or related fields
• Familiarity with cloud platforms (AWS, GCP, Azure) and their data services, with certifications preferred
• Experience with data quality frameworks (e.g., Great Expectations, Deequ)
• Experience with Git/GitHub for version control
• Knowledge of infrastructure-as-code tools (Terraform, CloudFormation)
• Exposure to containerization/orchestration (Docker, Kubernetes)
• Familiarity with data governance, lineage, and catalog tools (Apache Atlas, Amundsen)
• Experience with observability and monitoring tools (Monte Carlo, Datadog)
• Knowledge of machine learning pipelines
• Experience in trading or financial services environments

Interview Process

1. Review of CV by our partner and VP Engineering
2. First interview with our VP of Engineering
3. Additional technical and cultural fit interviews with team members or partners

We value cultural fit based on our core values: Drive, Ownership, Judgement, Openness, and Competence.

Benefits

Join a rapidly growing hedge fund managing a 9-figure AUM, with high returns and a collaborative team of about 40 professionals. Play a pivotal role in designing data infrastructures that enable research, analysis, and trading execution. If you are passionate about complex data systems, we want to hear from you!