Data Engineer - Global Hedge Fund - 300k+

Mondrian Alpha; New York (NY); On-site; USD 140,000 - 200,000; Full time

Job summary

Mondrian Alpha in New York City is seeking a data engineer to own scalable batch and streaming pipelines built on Databricks and Spark. You will work closely with researchers, risk managers, and portfolio teams to translate data needs into production-grade datasets and models.

You’ll be responsible for data quality, monitoring, and cost/performance optimization, while shaping how the platform evolves to support investment decisions.

Qualifications

• 2–10 years of data engineering experience.
• Strong Python, SQL, and Spark experience.
• Excellent ability to communicate with non engineers and stakeholders.

Responsibilities

• Design, build, and maintain large scale batch and streaming pipelines in Spark on Databricks.
• Own the full lifecycle of datasets from vendor onboarding to models and tables consumed by researchers.
• Collaborate with researchers, portfolio managers, and risk to translate data questions into concrete engineering work.
• Build monitoring, testing, and data quality controls to catch problems early.
• Improve performance and cost of Spark workloads and evolve the platform accordingly.
• Document and explain work to technical and non-technical audiences.

Skills

Python

SQL

Stakeholder communication

Cloud data platforms

Tools

Databricks

Spark

Airflow / orchestration tooling

Delta Lake / lakehouse

Job description

This is a data engineering role inside a global hedge fund where the data platform is not a support function, it is the thing the investment process runs on. The role owns pipelines that feed research, risk, and portfolio construction, which means the engineer sits in constant contact with the people actually making decisions with the output. Databricks and Spark are the core of the stack, and the work spans ingestion from vendors and exchanges through to the curated datasets researchers query every morning. If a pipeline fails silently and a dataset lands wrong, a portfolio manager makes a call on bad numbers before anyone notices. That is the standard the role is held to.

What You'll Do

• Design, build, and maintain large scale batch and streaming pipelines in Spark on Databricks
• Own the full lifecycle of datasets, from vendor onboarding and validation through to the models and tables researchers consume
• Work directly with researchers, portfolio managers, and risk to translate vague data questions into concrete engineering work
• Build the monitoring, testing, and data quality controls that catch problems before consumers do
• Improve performance and cost of existing Spark workloads, and make decisions about how the platform evolves
• Document and explain your work to technical and non technical audiences

Must-haves

• 2 to 10 years of data engineering experience
• Strong production experience with Databricks and Spark, including performance tuning
• Strong Python and SQL
• Demonstrated ability to communicate clearly with non engineers, defend technical decisions in plain language, and manage stakeholders directly. This is non-negotiable for the role, the engineer who cannot hold a conversation with a portfolio manager will not succeed here
• Experience with cloud data infrastructure (AWS, Azure, or GCP)

Nice-to-haves

• Exposure to financial or market data (tick data, reference data, corporate actions)
• Delta Lake, Unity Catalog, or lakehouse architecture experience
• Orchestration tooling (Airflow, Dagster, Databricks Workflows)
• Infrastructure as code and CI/CD for data pipelines
• Experience in a buy side or trading environment

Why This Role

The data platform here has direct, traceable impact on investment performance, and the engineers who build it are known by name to the people using it. The team is small enough that individual decisions shape the architecture rather than disappearing into a backlog, and the pace reflects a business where data problems are urgent by default. For a data engineer who is tired of building pipelines for people they never meet, and who wants their technical judgment to be visible to the desk, this is that.

Who should apply

Strong data engineers from top technology companies are actively encouraged to apply. No finance background required, but genuine curiosity about markets is expected.