Data Engineer Lead - AI-First Platform Architect

504 CGCG-US CG Companies Global-US; Los Angeles (CA); Hybrid; USD 202,000 - 323,000; Full time

Job summary

Capital Group is seeking a Data Engineer Lead in the Capital Solutions Group Technology family to set technical direction for data platforms powering portfolio construction and investment research. You will write code, own architecture, and lead cross‑functional delivery with investment, product, and tech leaders.

You will advance AI‑first engineering practices and build scalable, trustworthy data foundations.

Qualifications

• 10+ years in data or software engineering with technical leadership of complex production data platforms.
• Strong Python and SQL, with knowledge of distributed data processing and automated testing.
• Databricks on AWS including PySpark, Delta Lake, Unity Catalog, Databricks Jobs and SQL.
• Orchestrated Airflow pipelines (Astronomer) and dbt transformations with retries and backfills.
• Data modeling, governance incl. slowly changing dims, bi-temporal history and lineage.
• Data quality, observability, and CI/CD using Deequ, dbt tests, Datadog, Terraform, Harness.
• Experience preparing governed data for AI via Databricks Genie and related tools.
• Ability to define specs for AI agents, lead architecture discussions, and mentor engineers.
• Knowledge of prompt safety, audit trails, and enterprise data access controls.
• Bachelor’s in CS/Engineering or related field; willingness to lead and influence.

Responsibilities

• Set data engineering strategy and roadmap for the Lakehouse platform.
• Design and deliver ingestion, transformation, and serving pipelines with Databricks stack.
• Own complex, cross‑team data initiatives from requirements to production support.
• Lead AI‑first engineering evolution and define agent specifications and workflows.
• Build reusable patterns and patterns for data quality, testing, and monitoring.
• Collaborate with investment professionals and product managers on outcomes.
• Lead design reviews, guide day‑to‑day engineering activities, and mentor staff.

Skills

Python

SQL

Data engineering

Airflow

Databricks AWS

PySpark

Data quality

Leadership

AI tooling

Governance

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks

Unity Catalog

Delta Lake

dbt

Airflow

Astronomer

Databricks SQL

Databricks Asset Bundles

PostgreSQL

Job description

Capital Group is seeking a Data Engineer Lead in the Capital Solutions Group Technology family to set technical direction for data platforms powering portfolio construction and investment research. You will write code, own architecture, and lead cross‑functional delivery with investment, product, and tech leaders.

You will advance AI‑first engineering practices and build scalable, trustworthy data foundations.