Senior Data Platform Administrator

Jobtailor

Bengaluru

On-site

INR 1,400,000 - 2,200,000

Full time

6 days ago

Be an early applicant

Application generator

A complete application in a minute — tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Jobtailor in Bengaluru, India seeks a Databricks administrator to manage Databricks workspaces, clusters, jobs, and governance across data teams. The role focuses on performance tuning, cost optimization, and secure access controls while delivering robust CI/CD pipelines for notebooks and libraries.

You will partner with data engineers and scientists to troubleshoot issues and integrate Databricks with Azure Data Lake and enterprise systems, including BI tools, ensuring scalable platform

Qualifications

Production experience administering big data platforms.
Strong Spark and distributed computing knowledge.
Cloud platform experience (Azure preferred; AWS/GCP OK).
Familiar with Unity Catalog, Delta Lake, Lakehouse architec ture.
Proficient in Python, SQL and shell scripting; IaC/CI/CD.
Terraform is a must and excellent problem-solving skills.

Responsibilities

Handle and maintain workspaces, clusters, pools, and jobs in Databricks or other big data systems
Monitor performance, usage, and costs and optimize cluster configurations
Configure role-based access control and audit logging
Ensure compliance with data governance and security standards
Develop CI/CD pipelines for notebooks, jobs, and libraries using GitLab, GitHub Actions, Azure DevOps, and Terraform
Automate cluster lifecycle management and job scheduling
Support data scientists, engineers, and analysts using the platform
Troubleshoot job failures, performance bottlenecks, and integration issues
Integrate Databricks with Azure Data Lake, Power BI, Dataverse, and other enterprise systems
Collaborate with data engineering teams

Skills

Databricks Administration

Apache Spark

CI/CD Pipeline Development

Terraform

Cloud Platform Experience

Python

SQL

Shell Scripting

Data Governance Compliance

Role-Based Access Control

Job Scheduling Automation

Azure Data Lake Integration

Troubleshooting"

Tools

GitLab

GitHub Actions

Azure DevOps

Power BI

Dataverse

Job description

Handle and maintain workspaces, clusters, pools, and jobs in Databricks or other big data systems
Monitor performance, usage, and costs and optimize cluster configurations
Configure role-based access control and audit logging
Ensure compliance with data governance and security standards
Develop CI/CD pipelines for notebooks, jobs, and libraries using GitLab, GitHub Actions, Azure DevOps, and Terraform
Automate cluster lifecycle management and job scheduling
Support data scientists, engineers, and analysts using the platform
Troubleshoot job failures, performance bottlenecks, and integration issues
Integrate Databricks with Azure Data Lake, Power BI, Dataverse, and other enterprise systems
Collaborate with data engineering teams

Requirements

Proven experience administering big data platforms in a production environment
Strong understanding of Apache Spark and distributed computing
Experience with cloud platforms (Azure preferred; AWS or GCP acceptable)
Familiarity with Unity Catalog, Delta Lake, and Lakehouse architecture
Proficiency in Python, SQL, and shell scripting
Experience with CI/CD tools and infrastructure-as-code
Terraform is must
Excellent problem-solving and communication skills

Core Competencies

Demonstrates expertise in administering big data platforms, particularly in Databricks, with a strong focus on performance optimization, compliance with data governance, and security standards. Proficient in developing CI/CD pipelines and automating cluster management while collaborating effectively with data engineering teams.

Highest-signal resume keywords