Senior AI/ML Data Engineer — Scalable Data Pipelines

Blackline Systems Inc; New York (NY); On-site; USD 245,000 - 307,000; Full time

Benefits offered by this job

Incentive programs

Wellness plans

Hybrid work option

Pay transparency

Job summary

BlackLine is seeking a Lead Data Engineer to design, build, and optimize data pipelines powering AI-driven accounting agents. You will lead scalable, high‑performance data infrastructure and collaborate across teams to advance ML/AI workflows.

Responsibilities include building PySpark ETL pipelines, integrating client data sources, ensuring reliability, and driving observability. This role emphasizes leadership, governance, and cutting‑edge data tech.

Qualifications

• Experience designing and building scalable data infrastructure and AI/ML deployment pipelines.

Responsibilities

• Lead data pipeline development using PySpark ETL with emphasis on data quality and performance.
• Establish robust API/data source integrations (FiveTran, Plaid, internal connectors).
• Monitor pipelines, automate testing, and validate data accuracy.
• Implement scale improvements and governance standards for AI/ML workflows.
• Collaborate with stakeholders to refine data requirements and drive governance.

Skills

PySpark

ETL pipelines

Leadership

ML pipelines

CI/CD

Observability

Model deployment

Security & Compliance

API integration

Mentoring

Education

Bachelor’s or Master’s in CS/ML/DS

Tools

Airflow

Kubeflow

MLflow

LangChain

LangGraph

Docker

Kubernetes

Vertex AI

Prometheus

Grafana

New Relic

Job description

BlackLine is seeking a Lead Data Engineer to design, build, and optimize data pipelines powering AI-driven accounting agents. You will lead scalable, high‑performance data infrastructure and collaborate across teams to advance ML/AI workflows.

Responsibilities include building PySpark ETL pipelines, integrating client data sources, ensuring reliability, and driving observability. This role emphasizes leadership, governance, and cutting‑edge data tech.