Data Engineer Lead

504 CGCG-US CG Companies Global-US; Los Angeles (CA); Hybrid; USD 202,000 - 323,000; Full time

Job summary

Capital Group is seeking a Data Engineer Lead in the Capital Solutions Group Technology family to set technical direction for data platforms powering portfolio construction and investment research. You will write code, own architecture, and lead cross‑functional delivery with investment, product, and tech leaders.

You will advance AI‑first engineering practices and build scalable, trustworthy data foundations.

Qualifications

• 10+ years in data or software engineering with technical leadership of complex production data platforms.
• Strong Python and SQL, with knowledge of distributed data processing and automated testing.
• Databricks on AWS including PySpark, Delta Lake, Unity Catalog, Databricks Jobs and SQL.
• Orchestrated Airflow pipelines (Astronomer) and dbt transformations with retries and backfills.
• Data modeling, governance incl. slowly changing dims, bi-temporal history and lineage.
• Data quality, observability, and CI/CD using Deequ, dbt tests, Datadog, Terraform, Harness.
• Experience preparing governed data for AI via Databricks Genie and related tools.
• Ability to define specs for AI agents, lead architecture discussions, and mentor engineers.
• Knowledge of prompt safety, audit trails, and enterprise data access controls.
• Bachelor’s in CS/Engineering or related field; willingness to lead and influence.

Responsibilities

• Set data engineering strategy and roadmap for the Lakehouse platform.
• Design and deliver ingestion, transformation, and serving pipelines with Databricks stack.
• Own complex, cross‑team data initiatives from requirements to production support.
• Lead AI‑first engineering evolution and define agent specifications and workflows.
• Build reusable patterns and patterns for data quality, testing, and monitoring.
• Collaborate with investment professionals and product managers on outcomes.
• Lead design reviews, guide day‑to‑day engineering activities, and mentor staff.

Skills

Python

SQL

Data engineering

Airflow

Databricks AWS

PySpark

Data quality

Leadership

AI tooling

Governance

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks

Unity Catalog

Delta Lake

dbt

Airflow

Astronomer

Databricks SQL

Databricks Asset Bundles

PostgreSQL

Job description

I can be myself at work.

“I can be myself at work.” You are more than a job title. We want you to feel comfortable doing great work and bringing your best, authentic self to everything you do. We value your talents, traditions, and uniqueness—and we’re committed to fostering a strong sense of belonging in a respectful workplace. We intentionally seek diverse perspectives, experiences, and backgrounds, investing in a culture designed to celebrate differences. We believe that belonging leads to better outcomes and a stronger community of associates united by our mission. At Capital, we live our core values every day: Integrity, Client Focus, Diverse Perspectives, Long-Term Thinking, and Community.

I can influence my income.

You want to feel recognized at work. Your performance will be reviewed annually, and your compensation will be designed to motivate and reward the value that you provide. You’ll receive a competitive salary, bonuses and benefits. Your company-funded retirement contribution will factor in salary and variable pay, including bonuses.

I can lead a full life.

You bring unique goals and interests to your job and your life. Whether you’re raising a family, you’re passionate about where you volunteer, or you want to explore different career paths, we’ll give you the resources that can set you up for success. Enjoy generous time‑away and health benefits from day one, with the opportunity for flexible work options. Receive 2-for-1 matching gifts for your charitable contributions and the opportunity to secure annual grants for the organizations you love. Access on-demand professional development resources that allow you to hone existing skills and learn new ones.

I can succeed as a Data Engineer Lead at Capital Group.

As a Data Engineer Lead in Capital Solutions Group Technology (CSGT), you will set the technical direction for the data platform and data products that power portfolio construction, investment research, and monitoring capabilities for the Capital Solutions Group (CSG), the Capital Group investment unit that manages our multi-asset fund of funds. Our investment professionals rely on this data to construct and monitor portfolios, evaluate underlying funds, and make investment decisions. This is a senior, hands‑on individual contributor role. You will write code, own architecture and delivery, and partner closely with investment group, product management, and technology leaders. You will be a leader in our AI‑first engineering practices, where engineers direct AI agents through clear specifications, context, and verification. You will build the trusted data foundation that AI applications need to answer complex investment questions. You have an agile mindset and will lead the design, implementation, and delivery of large-scale, critical, and complex data architecture, storage, and pipelines. You are passionate about our mission, and committed to driving superior long‑term investment results through the application of modern engineering and data management methods.

You will:

• Set the data engineering strategy and roadmap for CSGT, including the Lakehouse architecture on Databricks and AWS.
• Make and explain decisions on scalability, security, reliability, and cost, and influence standards and practices across adjacent teams and the wider Capital Group technology organization, contributing to centers of excellence and firm‑wide engineering practices.
• Design and build ingestion, transformation, and serving pipelines with Databricks, PySpark, Delta Lake, dbt, and Airflow, and create reusable patterns and frameworks so the team delivers consistent, maintainable data products.
• Analyze new structured and unstructured datasets at the business‑capability level and fit them into the platform’s data domains and subject areas.
• Own delivery of complex, cross‑team data initiatives from requirements through production support, including estimates, work breakdown, sequencing, dependencies, and cost.
• Surface risks early and balance immediate business needs with a durable platform.
• Lead the continued evolution of AI‑first engineering: turn business outcomes into well‑defined specifications, engineer the business, architectural, and repository context agents need, and direct agents to plan, build, test, and document changes in small, reviewable increments.
• Build reusable agent workflows, skills, and tool integrations for data engineering work such as profiling, source‑to‑target mapping, pipeline and test generation, schema‑change analysis, and incident investigation.
• Define which actions agents can take on their own, which require human approval, and how agent activity is reviewed and traced.
• Make data understandable and reliable for AI through curated Unity Catalog metadata, lineage, business definitions, semantic models, and access controls, and connect it to Databricks Genie and other AI applications used by investment professionals.
• Define evaluation datasets and acceptance criteria for agents and AI‑generated SQL and code.
• Define the testing strategy across all layers of the platform, including performance, stability, and availability, and review and approve quality metrics before release.
• Build data quality, reconciliation, freshness, observability, and recovery controls into automated testing and CI/CD, with security and policy checks embedded from the start.
• Partner with investment professionals and product managers to develop a shared product vision and ownership of business outcomes, demonstrating how data and AI can scale research and portfolio construction.
• Raise the engineering bar: lead design and code reviews, direct the day‑to‑day work of engineers on your initiatives, and guide them through the most complex data and performance issues.
• Teach engineers to inspect and challenge AI‑generated work, share reusable patterns and context through internal and external forums, and help managers identify strengths and development needs.

Required qualifications:

• You have 10+ years of experience in data or software engineering, including technical leadership of complex production data platforms delivered across multiple teams.
• You have strong hands‑on Python and SQL skills, sound software design judgment, and a deep understanding of distributed data processing, query performance, and automated testing.
• You have production experience with Databricks on AWS, including PySpark, Delta Lake, Unity Catalog, Databricks Jobs, Databricks SQL, and Databricks Asset Bundles, and you understand the security, access, and cost implications of your designs.
• You have orchestrated production pipelines with Apache Airflow (including Astronomer) and built tested transformations with dbt, with reliable retries, backfills, and dependency management.
• You have strong data modeling and governance experience, including dimensional and time‑series models, slowly changing dimensions, bi‑temporal history, data contracts, lineage, and semantic metadata.
• You have implemented data quality, observability, and CI/CD for data platforms using tools such as Deequ, dbt tests, Lakehouse Monitoring, Datadog, Terraform, and Harness.
• You have experience preparing governed data for AI through natural‑language‑to‑SQL tools such as Databricks Genie, semantic metadata, or other governed data‑access patterns.
• You use AI coding agents well beyond code completion: writing specifications, supplying context, running tests, and reviewing generated changes through source control.
• You evaluate AI‑generated output with representative test cases, regression tests, execution traces, and human review, and you can distinguish a plausible answer from a verified one.
• You understand prompt injection, sensitive data handling, and least‑privilege access, and can design approval boundaries and audit trails for agents working against enterprise systems.
• You lead architecture discussions, influence without formal authority, develop other engineers, and explain technical choices and trade‑offs clearly to investment professionals and technology leaders.
• You are an agent of change with a sense of urgency: you question how work gets done and remove or automate what does not add value, while respecting what came before.
• You have a bachelor’s degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.

Preferred qualifications:

• You have experience with investment management data such as portfolios, positions, returns, exposures, benchmarks, and attribution, or with multi‑asset portfolio construction.
• You have experience building LLM applications or agent workflows that call tools and APIs, including context management, retrieval, state, and error handling.
• You have familiarity with Model Context Protocol (MCP).
• You have experience with PostgreSQL, SQL Server, or Lakebase, or with modernizing legacy data platforms onto a Lakehouse.

I can learn more about Capital Group.

At Capital Group, the success of the people who invest with us depends on the people in whom we invest. That’s why we offer a culture, compensation and opportunities that empower our associates to build successful and prosperous careers. Through nine decades, our goal is to improve people’s lives through successful investing. We know that our history is a testament to the strength of the people we hire. More than 9,000 associates in 30+ offices around the world help our clients and each other grow and thrive every day. Find us on LinkedIn, Instagram, YouTube and Glassdoor.

Southern California Base Salary Range: $201,683-$322,693

New York Base Salary Range: $213,795-$342,072

In addition to a highly competitive base salary, per plan guidelines, restrictions and vesting requirements, you also will be eligible for an individual annual performance bonus, plus Capital’s annual profitability bonus plus a retirement plan where Capital contributes 15% of your eligible earnings. You can learn more about our compensation and benefits here.

* Temporary positions in the United States are excluded from the above mentioned compensation and benefit plans.

We are an equal opportunity employer, which means we comply with all federal, state and local laws that prohibit discrimination when making all decisions about employment.

As equal opportunity employers, our policies prohibit unlawful discrimination on the basis of race, religion, color, national origin, ancestry, sex (including gender and gender identity), pregnancy, childbirth and related medical conditions, age, physical or mental disability, medical condition, genetic information, marital status, sexual orientation, citizenship status, AIDS/HIV status, political activities or affiliations, military or veteran status, status as a victim of domestic violence, assault or stalking or any other characteristic protected by federal, state or local law.

Capital Group is one of the largest and most trusted financial companies in the world, with the goal of improving people's lives through successful investing.

Our shared values of integrity, client focus, long‑term view, diverse perspectives and community inspire our associates every day. A career at Capital Group means having countless opportunities to explore, grow and succeed. Learn more about compensation & retirement Learn more about our benefits Learn more about our hybrid schedule Learn more about our culture and core values Learn more about our commitment to responsible AI and how it shapes your candidate experience