Databricks Platform Lead - AWS & CI/CD

Jobtailor; New York (NY); On-site; USD 140,000 - 190,000; Full time

Job summary

Jobtailor in New York seeks an experienced Databricks Platform Administrator to design, secure, and operate the enterprise Databricks environment on AWS. You will manage multiple workspaces, configure Unity Catalog and IAM integration, and drive CI/CD, Git integration, and monitoring.

You will own runtime version upgrades, cluster governance, cost optimization, and incident resolution while collaborating with security and DevOps teams to maintain a stable, scalable platform.

Qualifications

• 5–10 years of overall IT experience.
• 3–5+ years administering Databricks platforms in production on AWS.
• Experience with Databricks Platform, workspace administration, and Unity Catalog.
• Strong IAM, cluster, runtime management, and Terraform capabilities.
• Background in DevOps, CI/CD, Git, and security practices.

Responsibilities

• Design, administer, secure, and optimize the enterprise Databricks platform on AWS.
• Manage enterprise Databricks platforms and multiple workspaces.
• Configure Unity Catalog and IAM integration.
• Create and govern cluster policies and instance pools.
• Upgrade runtime versions and implement CI/CD with Git integration.
• Monitor platform performance, troubleshoot issues, and manage secrets.
• Support production operations and drive cost optimization initiatives.

Skills

Databricks Platform Administration

AWS IAM Integration

CI/CD Implementation

Cluster Management

Cost Optimization

Security Best Practices

Performance Tuning

Incident Management

Log Analysis

Git

Terraform

Infrastructure as Code

DevOps

Tools

Databricks Platform

Terraform

Azure DevOps

GitHub Actions

Jenkins

CloudWatch

KMS

Job description

Jobtailor in New York seeks an experienced Databricks Platform Administrator to design, secure, and operate the enterprise Databricks environment on AWS. You will manage multiple workspaces, configure Unity Catalog and IAM integration, and drive CI/CD, Git integration, and monitoring.

You will own runtime version upgrades, cluster governance, cost optimization, and incident resolution while collaborating with security and DevOps teams to maintain a stable, scalable platform.