Senior Data Engineer

Goldman Lloyds; New York (NY); On-site; USD 150,000 - 210,000; Full time

Job summary

Goldman Lloyds is seeking a Senior Data Engineer to build and evolve data infrastructure used by PMs, quants, and investment teams.

You will design scalable Python ETL pipelines, develop cloud-based data lakes on AWS and Databricks, and ensure data quality and integrity for research, portfolio construction, and decision making.

Qualifications

• Buyside experience is mandatory.
• Advanced Python and SQL skills.
• Experience building production-grade data pipelines.
• Strong knowledge of modern data architecture, APIs, databases and cloud technologies.
• Experience with financial market or investment datasets is preferred.

Responsibilities

• Design, develop, and support scalable Python ETL pipelines to ingest and transform market data.
• Build and enhance cloud-based data lake infrastructure using AWS and Databricks.
• Establish robust data validation and quality frameworks.
• Optimize data workflows for high throughput and low latency for research and trading.
• Collaborate with Portfolio Managers, Quants, and Traders to deliver data solutions.
• Document architecture, data lineage, workflows, and implementations.

Skills

Python

SQL

Data pipelines

Data architecture

APIs

Databases

Cloud technologies

AWS

Snowflake

Databricks

Spark

Kafka

Tools

Python

SQL

AWS

Snowflake

Databricks

Spark

Kafka

Job description

*Buyside experience is mandatory for this seat.

A leading hedge fund is seeking a Senior Data Engineer to build and evolve the data infrastructure used directly by Portfolio Managers, Quantitative Researchers and investment teams. This is an investment-facing engineering role where you'll work closely with PMs to understand how they consume data, identify new datasets and build scalable solutions that directly support research, portfolio construction and investment decision-making.

Key Responsibilities

• Design, develop, and support scalable Python ETL pipelines to ingest, process, and transform market data from a variety of internal and external sources.
• Build and enhance cloud-based data lake infrastructure, leveraging AWS and Databricks to manage large-scale structured and unstructured datasets.
• Establish robust data validation, quality control, and cleansing frameworks to maintain the integrity and reliability of financial data.
• Engineer and optimize data workflows for high throughput and low-latency performance, supporting quantitative research, trading strategies, and risk analytics.
• Partner closely with Portfolio Managers, Quantitative Researchers, and Traders to deliver data solutions supporting modelling, strategy research, and investment decision-making.
• Participate in the design, development, and ongoing enhancement of the firm's security master.
• Maintain comprehensive documentation covering system architecture, data lineage, workflows, and technical implementations to support transparency and reproducibility.

Candidate Profile

• Strong professional experience in Data Engineering, ideally within a hedge fund, asset manager, trading firm or investment bank.
• Advanced Python and SQL skills.
• Experience designing and building production-grade data pipelines and platforms.
• Strong knowledge of modern data architecture, APIs, databases and cloud technologies.
• Experience working with financial market, investment or alternative datasets.
• Ability to work directly with Portfolio Managers, Quant Researchers and other demanding front-office users.
• Strong software engineering fundamentals with an emphasis on reliability, scalability and data quality.
• Experience with technologies such as AWS, Snowflake, Databricks, Spark, Kafka or similar is advantageous.

Why Join?

This isn’t a traditional back-office data engineering role. You’ll sit close to the investment process, working directly with Portfolio Managers to solve data problems that influence research and investment decisions.

For a strong Data Engineer looking for greater front-office exposure, ownership and direct investment impact, this offers a compelling opportunity within a sophisticated hedge fund environment.