Databricks Platform Engineer - Azure, IaC & DevOps

KPG99 INC; Quincy (MA); Hybrid; USD 140,000 - 190,000; Full time

Job summary

KPG99 INC is seeking a Senior Platform Engineer for a Hybrid role in Quincy, MA. You will design, deploy, and manage Databricks workspaces across multiple Azure environments, implementing IaC with Terraform and ensuring security and governance across platforms.

You will build CI/CD pipelines with Azure DevOps, automate testing of data pipelines, and mentor teammates on Databricks best practices. In-person interview is required; this is a full-time direct hire.

Qualifications

• 7+ years of experience in platform engineering, infrastructure engineering, or DevOps.
• Strong expertise with Databricks infrastructure setup or Spark/Delta Lake environments.
• 3+ years of experience with Azure cloud services and Azure DevOps Pipelines, Repos, Artifacts.

Responsibilities

• Design, deploy, and manage Databricks workspaces across Azure subscriptions and environments (dev, staging, production).
• Implement IaC solutions using Terraform for Databricks resources.
• Establish security, including RBAC, network isolation, and data governance.
• Configure cluster management, auto-scaling, and cost optimization.
• Manage workspace connectivity and networking (VNets, private endpoints, firewalls).
• Build CI/CD pipelines using Azure DevOps for notebooks, jobs, and configurations.
• Automate testing and validation of data pipelines and create self-service deployment frameworks.
• Establish monitoring, alerting, and logging for Databricks environments.
• Automate workspace provisioning/deprovisioning; mentor team on best practices.

Skills

Infrastructure engineering

Platform engineering

Security best practices

Education

Bachelor of Computer Science

Tools

Terraform Enterprise

Azure DevOps

Databricks

Docker

Kubernetes

Job description

KPG99 INC is seeking a Senior Platform Engineer for a Hybrid role in Quincy, MA. You will design, deploy, and manage Databricks workspaces across multiple Azure environments, implementing IaC with Terraform and ensuring security and governance across platforms.

You will build CI/CD pipelines with Azure DevOps, automate testing of data pipelines, and mentor teammates on Databricks best practices. In-person interview is required; this is a full-time direct hire.