Azure DataOps Engineer – Databricks & CI/CD

Monogram Health; United States; Hybrid; USD 138,000 - 182,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

401k with employer match

20 PTO days, 7 holidays, 1 floatingday

Wellness & Growth resources

Job summary

Monogram Health is seeking a DevOps Engineer in the United States to build and maintain CI/CD pipelines for notebooks, jobs, and workflows in Azure Databricks. You will integrate Databricks with Git, automate notebook execution, and uphold infrastructure-as-code using Terraform while securing secrets in Azure Key Vault.

You will collaborate with Data Engineers and Platform teams to manage dev, staging, and production, ensuring robust version control, testing, and notebook promotions across

Qualifications

• Bachelor’s degree in computer science, information technology, or related field or equivalent experience.
• Minimum of three (3) years in DevOps engineering or related role with Azure focus.

Responsibilities

• Develop and maintain CI/CD pipelines in Azure DevOps to deploy notebooks, jobs, and workflows to Azure Databricks.
• Integrate Databricks Repos with Azure Repos (Git) for source control of PySpark notebooks.
• Automate notebook/job runs in Databricks via Databricks CLI, REST API, or DevOps tasks.
• Maintain infrastructure-as-code using Terraform.
• Monitor and troubleshoot pipeline failures, notebook issues, and deployment inconsistencies.
• Collaborate with Data Engineers and Platform teams to manage development, staging, and production environments.
• Enforce best practices for version control, testing, and notebook promotion across environments.
• Securely manage Databricks secrets, tokens, and configurations in Azure Key Vault and DevOps groups.

Skills

CI/CD pipelines

Azure DevOps

Databricks

Git

Python scripting

Terraform

Azure Key Vault

Automation

Education

Bachelor’s degree in CS/IT

Tools

Databricks CLI

REST API

Job description

Monogram Health is seeking a DevOps Engineer in the United States to build and maintain CI/CD pipelines for notebooks, jobs, and workflows in Azure Databricks. You will integrate Databricks with Git, automate notebook execution, and uphold infrastructure-as-code using Terraform while securing secrets in Azure Key Vault.

You will collaborate with Data Engineers and Platform teams to manage dev, staging, and production, ensuring robust version control, testing, and notebook promotions across