AI-First Data Engineer Lead

Capital Group; New York (NY); On-site; USD 180,000 - 260,000; Full time

Benefits offered by this job

Generous time-off and health benefits

Company-funded retirement contribution

2-for-1 charitable gifts matching

Job summary

Capital Group seeks a Senior Data Engineer Lead to set a technical direction for data platforms powering portfolio construction and investment research. You will write code, own architecture, and partner with investment, product, and technology leaders in an AI-first engineering approach.

You will design scalable ingestion, transformation, and serving pipelines, influence standards, and lead cross-team data initiatives across Databricks on AWS with a focus on reliability, security, and cost.

Qualifications

• Senior, hands-on data engineer lead role leading platform and data product delivery.
• Experience designing data pipelines, storage, and governance at scale.
• Strong collaboration with investment, product, and technology teams.

Responsibilities

• Set the data engineering strategy and roadmap, including Lakehouse architecture on Databricks and AWS.
• Design and build ingestion, transformation, and serving pipelines with Databricks, PySpark, Delta Lake, dbt, and Airflow.
• Analyze structured and unstructured datasets and map them to data domains.
• Own delivery of complex, cross-team data initiatives from requirements to production support.
• Lead AI-first engineering efforts by turning business outcomes into well-defined specs for agents and data products.
• Create reusable agent workflows and tool integrations for data engineering tasks.
• Ensure data understandability and reliability for AI with metadata, lineage, and semantic models.

Skills

Databricks

PySpark

Delta Lake

dbt

Airflow

AI-first engineering

Data architecture

Tools

Databricks

PySpark

Delta Lake

dbt

Airflow

Job description

Capital Group seeks a Senior Data Engineer Lead to set a technical direction for data platforms powering portfolio construction and investment research. You will write code, own architecture, and partner with investment, product, and technology leaders in an AI-first engineering approach.

You will design scalable ingestion, transformation, and serving pipelines, influence standards, and lead cross-team data initiatives across Databricks on AWS with a focus on reliability, security, and cost.