Databricks Platform Engineer — Azure, Automation & Governance

Robotics Prcocess Automation, LLC; St. Louis (MO); On-site; USD 120,000 - 160,000; Full time

Job summary

ROBOTIC PROCESS AUTOMATION LLC seeks a skilled Databricks Platform Engineer to design, implement, and manage scalable Databricks environments on Azure. You will build and maintain workspaces, clusters, and data pipelines, while automating provisioning with Terraform, CI/CD, and scripting.

Collaborating with data scientists and ML engineers, you will monitor health, optimize compute/storage, and enforce security policies using Unity Catalog and related features.

Qualifications

• Experience architecting, deploying, managing, and supporting Azure Databricks Platform and Azure Data Lake solutions.
• Hands-on with scripting/CLI tools (Ansible, Terraform, Azure CLI, Databricks CLI, PowerShell or Bash).
• Preferred experience with Azure ADLS storage and Databricks tuning.
• Knowledge of CI/CD pipelines and Git versioning.
• Exposure to data ingestion/pipeline tools (Qlik Replicate, Qlik Compose, Databricks, Kafka, Matillion ELT, Snowflake).
• Experience in an Agile product development team.

Responsibilities

• Design, implement, and manage scalable Databricks platforms for performance and reliability.
• Create and manage Databricks workspaces and clusters with proper lifecycle management.
• Automate provisioning with IaC tools (Terraform, CI/CD pipelines) and scripting.
• Monitor platform health, performance, and cost with dashboards and monitoring integrations.
• Troubleshoot cluster configuration, job failures, and performance bottlenecks.

Skills

Databricks Data Engineering

Azure Databricks

Communication skills

Agile teamwork

Problem solving

Tools

Ansible

SQL

Terraform

Azure CLI

Databricks CLI

PowerShell

Bash

Git

Job description

ROBOTIC PROCESS AUTOMATION LLC seeks a skilled Databricks Platform Engineer to design, implement, and manage scalable Databricks environments on Azure. You will build and maintain workspaces, clusters, and data pipelines, while automating provisioning with Terraform, CI/CD, and scripting.

Collaborating with data scientists and ML engineers, you will monitor health, optimize compute/storage, and enforce security policies using Unity Catalog and related features.