Staff II AI/ML Operations Engineer - Data Pipelines & Scale

BlackLine; New York (NY); Hybrid; USD 245,000 - 307,000; Full time

Job summary

BlackLine is seeking a Lead Data Engineer to design, build, and optimize data pipelines powering AI-driven accounting agents. You will lead scalable, high-performance data infrastructure and collaborate across teams to advance AI and big data technologies.

Responsibilities include leading PySpark ETL pipelines, API integrations (FiveTran, Plaid), data quality, and scale-focused optimizations. Mentoring engineers and driving ML/AI deployment readiness are key parts of the role.

Qualifications

• Bachelor's or Master's degree in CS, ML, Data Science, or related field.
• Proven leadership and data-engineering experience.
• Strong programming in Python/Scala/Java; solid ML/AI tooling knowledge.

Responsibilities

• Lead data pipeline development with PySpark ETL pipelines.
• Establish robust data source integrations via APIs and tools (FiveTran, Plaid, internal connectors).
• Ensure pipeline reliability through monitoring, testing, and data validation.
• Scale data infrastructure with CDC, indexing, and performance optimizations.
• Collaborate with stakeholders to refine data requirements and AI/big data tech adoption.
• Define and govern model registries, agent runtimes, and ML deployment processes.

Skills

Leadership

Data engineering

Python

PySpark

CI/CD

ML/AI systems

Observability

Governance

Mentor engineers

Communication

Education

Bachelor's or Master's degree

Tools

PySpark

Airflow

Kubeflow

MLflow

LangChain

LangGraph

APIs

FiveTran

Plaid

Kubernetes

Job description

BlackLine is seeking a Lead Data Engineer to design, build, and optimize data pipelines powering AI-driven accounting agents. You will lead scalable, high-performance data infrastructure and collaborate across teams to advance AI and big data technologies.

Responsibilities include leading PySpark ETL pipelines, API integrations (FiveTran, Plaid), data quality, and scale-focused optimizations. Mentoring engineers and driving ML/AI deployment readiness are key parts of the role.