Lead Data Engineer: AI/ML Pipelines & Infra

BlackLine Systems

Pleasanton (CA)

On-site

USD 245,000 - 307,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

BlackLine seeks a Lead Data Engineer to design, build, and optimize data pipelines powering our AI-driven accounting agents. You will lead scalable, high-performance data infrastructure while collaborating closely across teams.

Responsibilities include leading PySpark ETL pipelines, integrating client data via APIs and tools like FiveTran and Plaid, and ensuring data quality, reliability, and scale. You will help advance ML-enabled workflows across the org.

Qualifications

  • Proven experience building data pipelines with strong data quality and performance.
  • Experience leading data pipeline development and cross-functional collaboration.
  • Ability to design scalable data infra and integrate AI/ML workflows.

Responsibilities

  • Lead data pipeline development: Build and maintain PySpark ETL pipelines with high data quality and performance.
  • Manage integrations: Establish robust connections to client data sources via APIs and tools like FiveTran, Plaid, and BlackLine’s internal connectors.
  • Ensure reliability: Monitor pipeline performance, automate testing, and validate data accuracy.
  • Optimize for scale: Implement performance improvements (e.g., CDC mechanisms, indexing strategies) for large-scale datasets.
  • Collaborate across teams to refine data requirements and advance ML/AI workloads.

Skills

Python
Java
Scala
PySpark
ETL design

Education

Bachelor's or Master's in CS/ML/Data Science

Tools

FiveTran
Plaid
Internal connector ecosystem

Job description

BlackLine seeks a Lead Data Engineer to design, build, and optimize data pipelines powering our AI-driven accounting agents. You will lead scalable, high-performance data infrastructure while collaborating closely across teams.

Responsibilities include leading PySpark ETL pipelines, integrating client data via APIs and tools like FiveTran and Plaid, and ensuring data quality, reliability, and scale. You will help advance ML-enabled workflows across the org.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead AI/ML Data Engineer: Scalable Pipelines & Ops
Lead AI/ML Data Engineer: Scalable Pipelines & Ops

BlackLine Systems, Inc. • Seattle (WA)

Hybrid
USD 245,000 - 307,000
Lead Data Engineer for AI & ML Pipelines
Lead Data Engineer for AI & ML Pipelines

BlackLine, Inc. • Los Angeles (CA)

On-site
USD 245,000 - 307,000
Lead Data Engineer: PySpark & AI Pipelines
Lead Data Engineer: PySpark & AI Pipelines

BlackLine, Inc. • New York (NY)

On-site
USD 193,000 - 242,000
Staff II AI/ML Operations Engineer - Data Pipelines & Scale
Staff II AI/ML Operations Engineer - Data Pipelines & Scale

BlackLine, Inc. • New York (NY)

Hybrid
USD 245,000 - 307,000
Lead Data Engineer: Scalable Pipelines & AI-Driven Insights
Lead Data Engineer: Scalable Pipelines & AI-Driven Insights

Code and Theory, LLC • Chicago (IL)

On-site
USD 120,000 - 160,000
Staff Data Engineer
Staff Data Engineer

BlackLine, Inc. • New York (NY)

On-site
USD 193,000 - 242,000
Lead Data Engineer – Lakehouse & ML Pipelines
Lead Data Engineer – Lakehouse & ML Pipelines

CloudBigD • Lansing (MI)

On-site
USD 87,000 - 114,000
Lead Data Engineer — AI/ML Pipelines (Remote, US)
Lead Data Engineer — AI/ML Pipelines (Remote, US)

3M • United States

Remote
USD 188,000 - 230,000
Medical, Dental & Vision
Health Savings Accounts
Disability Benefits
+3
Lead Data Engineer — AI-Driven Pipelines & Cloud Platforms
Lead Data Engineer — AI-Driven Pipelines & Cloud Platforms

Code and Theory, LLC • New York (NY)

On-site
USD 120,000 - 160,000
Senior Data Engineer: Scalable Pipelines & Analytics
Senior Data Engineer: Scalable Pipelines & Analytics

Black Financial Consulting Group • Snowflake (AZ)

On-site
USD 120,000 - 160,000