Remote DataOps Engineer: AI-Observability & Databricks

Ladders; United States; Remote; USD 120,000 - 150,000; Full time

Benefits offered by this job

Medical, dental, vision, disability, &

Life insurance

PTO: 15 vacation days + 11 holidays

Sabbatical after 10 years

401(k) with 6% company match

Parental leave & fertility benefits

Employee Assistance Program

Charitable giving program

Job summary

Ladders is seeking a DataOps Engineer to own the data platform operations, focusing on observability, reliability, and scalable delivery. The role partners with data engineering and platform teams to improve data quality and reduce MTTR using AI-enabled tooling.

You'll implement and optimize Databricks pipelines, Azure Data Factory workflows, and CI/CD processes in a production environment, with remote work options in the United States.

Qualifications

• 7+ years of DataOps, data engineering, or platform engineering experience.
• Proficient with Databricks including Delta Live Tables, Jobs/Workflows, Unity Catalog, and Spark tuning.
• Strong experience with Azure data services: ADF, ADLS Gen2, Azure Monitor, Log Analytics, Key Vault.
• Experience using AI tools in DataOps or observability workflows.
• Proficient in CI/CD with GitHub Enterprise.
• Proficient in Python and/or Scala; SQL for data transformation.
• Hands-on experience with ADF pipeline design and orchestration at scale.

Responsibilities

• Implement AI-powered observability to detect pipeline drift and classify anomalies.
• Build monitoring workflows that proactively surface data quality issues.
• Integrate AI tooling into operational DataOps processes as production-grade capabilities.
• Develop AI-assisted root cause analysis tooling to reduce MTTR on pipeline failures.
• Operate the full Azure data services stack supporting the data platform.
• Design and maintain ADF pipelines for source system ingestion.
• Own the design, build, and optimization of data pipelines on Databricks.

Skills

Databricks

Azure

CI/CD

Python

SQL

Scala

GitHub Enterprise

Observability

AI in DataOps

Tools

Delta Live Tables

Unity Catalog

ADF

ADLS Gen2

Azure Monitor

Log Analytics

Key Vault

Job description

Ladders is seeking a DataOps Engineer to own the data platform operations, focusing on observability, reliability, and scalable delivery. The role partners with data engineering and platform teams to improve data quality and reduce MTTR using AI-enabled tooling.

You'll implement and optimize Databricks pipelines, Azure Data Factory workflows, and CI/CD processes in a production environment, with remote work options in the United States.