Senior Databricks Platform Engineer

Quantum Technologies. LLC; Weehawken (NJ); On-site; USD 83,000 - 131,000; Full time

Job summary

Quantum Technologies. LLC is seeking a hands-on Senior Databricks Platform Administrator to manage enterprise Databricks environments, focusing on Unity Catalog, permissions, and data security within cloud infrastructure.

The role emphasizes governance, automation, performance optimisation, and cost monitoring, with collaboration across cross-functional teams using Python, SQL, Terraform, and CI/CD in Azure or AWS ecosystems.

Qualifications

• 5+ years of platform administration or cloud engineering experience.
• 3+ years of hands-on Databricks administration experience.
• Strong expertise with Unity Catalog, Databricks Compute, SQL Warehouses, IAM/RBAC.
• Experience with Python, SQL, REST APIs, Azure or AWS, Terraform and CI/CD.
• Experience with Apache Spark and Delta Lake.
• Networking, SSO, OAuth and identity integrations.

Responsibilities

• Administer Databricks workspaces, clusters, Unity Catalog, permissions, and access controls.
• Manage storage credentials, external locations, catalogs, and data security policies.
• Support Databricks compute, SQL Warehouses, and serverless environments.
• Perform platform upgrades, patching, and environment maintenance.
• Troubleshoot platform, networking, authentication, and performance issues.
• Integrate Databricks with Azure/AWS services, databases, APIs, and external systems.
• Automate administrative processes using Python, SQL, APIs, Terraform, and CI/CD tools.
• Monitor platform health, utilization, performance, and cost optimization.

Skills

Databricks admin

Cloud engineering

Platform security

Python/SQL

Terraform/CI-CD

Azure/AWS

Tools

Unity Catalog

Databricks Compute

SQL Warehouses

IAM/RBAC

Terraform

CI/CD

APIs

Delta Lake

Apache Spark

Job description

Quantum Technologies. LLC is seeking a hands-on Senior Databricks Platform Administrator to manage enterprise Databricks environments, focusing on Unity Catalog, permissions, and data security within cloud infrastructure.

The role emphasizes governance, automation, performance optimisation, and cost monitoring, with collaboration across cross-functional teams using Python, SQL, Terraform, and CI/CD in Azure or AWS ecosystems.