Data Engineer Lead

Capital Group; New York (NY); On-site; USD 180,000 - 260,000; Full time

Benefits offered by this job

Generous time-off and health benefits

Company-funded retirement contribution

2-for-1 charitable gifts matching

Job summary

Capital Group seeks a Senior Data Engineer Lead to set a technical direction for data platforms powering portfolio construction and investment research. You will write code, own architecture, and partner with investment, product, and technology leaders in an AI-first engineering approach.

You will design scalable ingestion, transformation, and serving pipelines, influence standards, and lead cross-team data initiatives across Databricks on AWS with a focus on reliability, security, and cost.

Qualifications

• Senior, hands-on data engineer lead role leading platform and data product delivery.
• Experience designing data pipelines, storage, and governance at scale.
• Strong collaboration with investment, product, and technology teams.

Responsibilities

• Set the data engineering strategy and roadmap, including Lakehouse architecture on Databricks and AWS.
• Design and build ingestion, transformation, and serving pipelines with Databricks, PySpark, Delta Lake, dbt, and Airflow.
• Analyze structured and unstructured datasets and map them to data domains.
• Own delivery of complex, cross-team data initiatives from requirements to production support.
• Lead AI-first engineering efforts by turning business outcomes into well-defined specs for agents and data products.
• Create reusable agent workflows and tool integrations for data engineering tasks.
• Ensure data understandability and reliability for AI with metadata, lineage, and semantic models.

Skills

Databricks

PySpark

Delta Lake

dbt

Airflow

AI-first engineering

Data architecture

Tools

Databricks

PySpark

Delta Lake

dbt

Airflow

Job description

"I can be myself at work."

You are more than a job title. We want you to feel comfortable doing great work and bringing your best, authentic self to everything you do. We value your talents, traditions, and uniqueness-and we're committed to fostering a strong sense of belonging in a respectful workplace.

We intentionally seek diverse perspectives, experiences, and backgrounds, investing in a culture designed to celebrate differences. We believe that belonging leads to better outcomes and a stronger community of associates united by our mission. At Capital, we live our core values every day: Integrity, Client Focus, Diverse Perspectives, Long-Term Thinking, and Community.

"I can influence my income."

You want to feel recognized at work. Your performance will be reviewed annually, and your compensation will be designed to motivate and reward the value that you provide. You'll receive a competitive salary, bonuses and benefits. Your company-funded retirement contribution will factor in salary and variable pay, including bonuses.

"I can lead a full life."

You bring unique goals and interests to your job and your life. Whether you're raising a family, you're passionate about where you volunteer, or you want to explore different career paths, we'll give you the resources that can set you up for success.

• Enjoy generous time-away and health benefits from day one, with the opportunity for flexible work options

• Receive 2-for-1 matching gifts for your charitable contributions and the opportunity to secure annual grants for the organizations you love

• Access on-demand professional development resources that allow you to hone existing skills and learn new ones

"I can succeed as a Data Engineer Lead at Capital Group."

As a Data Engineer Lead in Capital Solutions Group Technology (CSGT), you will set the technical direction for the data platform and data products that power portfolio construction, investment research, and monitoring capabilities for the Capital Solutions Group (CSG), the Capital Group investment unit that manages our multi-asset fund of funds. Our investment professionals rely on this data to construct and monitor portfolios, evaluate underlying funds, and make investment decisions. This is a senior, hands‑on individual contributor role. You will write code, own architecture and delivery, and partner closely with investment group, product management, and technology leaders. You will be a leader in our AI‑first engineering practices, where engineers direct AI agents through clear specifications, context, and verification. You will build the trusted data foundation that AI applications need to answer complex investment questions. You have an agile mindset and will lead the design, implementation, and delivery of large‑scale, critical, and complex data architecture, storage, and pipelines. You are passionate about our mission, and committed to driving superior long‑term investment results through the application of modern engineering and data management methods.

You will:

• Set the data engineering strategy and roadmap for CSGT, including the Lakehouse architecture on Databricks and AWS. Make and explain decisions on scalability, security, reliability, and cost, and influence standards and practices across adjacent teams and the wider Capital Group technology organization, contributing to centers of excellence and firm‑wide engineering practices.

• Design and build ingestion, transformation, and serving pipelines with Databricks, PySpark, Delta Lake, dbt, and Airflow, and create reusable patterns and frameworks so the team delivers consistent, maintainable data products.

• Analyze new structured and unstructured datasets at the business‑capability level and fit them into the platform's data domains and subject areas.

• Own delivery of complex, cross‑team data initiatives from requirements through production support, including estimates, work breakdown, sequencing, dependencies, and cost. Surface risks early and balance immediate business needs with a durable platform.

• Lead the continued evolution of AI‑first engineering: turn business outcomes into well‑defined specifications, engineer the business, architectural, and repository context agents need, and direct agents to plan, build, test, and document changes in small, reviewable increments.

• Build reusable agent workflows, skills, and tool integrations for data engineering work such as profiling, source‑to‑target mapping, pipeline and test generation, schema‑change analysis, and incident investigation. Define which actions agents can take on their own, which require human approval, and how agent activity is reviewed and traced.

• Make data understandable and reliable for AI through curated Unity Catalog metadata, lineage, business definitions, semantic models, and access controls, and connect it to Databricks Genie and other AI applications used by investment professionals. Define evaluation datasets and acceptance criteria for agents and AI‑