Senior AI/ML Engineer: Scalable Data Pipelines

BlackLine Systems Inc (U.S.); Pleasanton (TX); On-site; USD 145,000 - 182,000; Full time

Job summary

BlackLine is seeking a Senior AI/ML Engineer to design, build, and optimize data pipelines powering AI-driven accounting agents. You will lead scalable data infrastructure, collaborating across teams to integrate AI and big data technologies, ensuring reliability and performance in production environments.

The role requires 3+ years in Python/Java/Scala, strong ML framework experience, and proficiency with cloud- ML workflows.

Qualifications

• 3+ years of experience with programming in Python, Java, or Scala.
• Expertise in ML frameworks (TensorFlow, PyTorch, scikit-learn) and orchestration tools (Airflow, Kubeflow, Vertex AI, MLflow).
• Proven experience operating production ML/LLM pipelines across cloud providers (GCP, AWS, Azure).
• Familiarity with LangChain, LangGraph, and actor runtimes.
• Strong CI/CD, IaC, and DevSecOps for automated deployment and testing.
• Hands-on with observability stacks (Prometheus, Grafana, New Relic).
• Understanding governance for Responsible AI, auditability and cost metering.
• Proficiency with Docker and Kubernetes in production.

Responsibilities

• Lead data pipeline development: build and maintain PySpark ETL pipelines with quality and performance.
• Manage integrations to client data sources via APIs and tools (FiveTran, Plaid, internal connectors).
• Ensure reliability: monitor pipelines, automate tests, verify data accuracy.
• Optimize for scale: implement CDC, indexing, and performance enhancements for large datasets.
• Collaborate with stakeholders to refine data requirements and apply AI/big data tech.
• Lead incident response and reliability strategies for ML/AI systems.
• Integrate AI solutions into existing workflows and applications.
• Define and manage model/agent registries and governance for multi-agent runtimes.
• Build CI/CD pipelines automating model deployment, policy validation, and prompts evaluation.
• Develop experimentation frameworks for agent evaluations and performance analytics.
• Implement logging, metering, auditing for agent behavior and compliance.
• Create observability systems tracking outcomes, latency, escalation patterns, and safety events.
• Architect guardrails for AI agents including prompt injection protection and access controls.
• Standardize authentication, authorization, and session governance across multi-agent runtimes.
• Design model registries and feature stores for versioning and reproducibility across environments.
• Lead deployment of ML models into production with scalability and reliability.
• Collaborate with software engineers to integrate models into applications.
• Implement APIs for model inference and manage training infrastructure (distributed training, GPU/TPU resources).
• Implement CI/CD for model workflows with validation and bias checks; enable rollback.

Skills

Python/Java/Scala

ML frameworks

Production ML pipelines

LangChain/agent runtimes

CI/CD & IaC

Observability & monitoring

Governance & compliance

Containerization (Docker/Kubernetes)

Tools

Airflow

Kubeflow

Vertex AI

MLflow

LangChain

LangGraph

Docker

Kubernetes

Job description

BlackLine is seeking a Senior AI/ML Engineer to design, build, and optimize data pipelines powering AI-driven accounting agents. You will lead scalable data infrastructure, collaborating across teams to integrate AI and big data technologies, ensuring reliability and performance in production environments.

The role requires 3+ years in Python/Java/Scala, strong ML framework experience, and proficiency with cloud- ML workflows.