Databricks Platform Administrator

Central Business Solutions, Inc; Bengaluru; On-site; INR 800,000 - 1,200,000; Full time

Job summary

Central Business Solutions, Inc is seeking a Databricks platform administrator in Bengaluru to manage Databricks clusters and workspaces, configure deployments, and support development teams in an Agile/DevOps setting.

The role requires cloud experience, familiarity with Terraform and AWS resources, and a focus on security, cost optimization, backup strategies, and cross-functional collaboration with data engineers and analysts.

Qualifications

• Experience administering Databricks platforms and clusters.
• Experience with cloud platforms and AWS resources.
• Familiarity with Agile/DevOps methodologies.
• Ability to configure, deploy, and maintain Databricks clusters.

Responsibilities

• Manage Databricks platform, clusters and workspaces.
• Collaborate with data engineers to integrate Databricks with data sources, warehouses and lakes.
• Support application development teams, assist debugging and issue resolution.
• Implement security policies, RBAC and encryption mechanisms.
• Monitor performance and optimize cluster configurations for cost and performance.

Skills

Cloud experience

Terraform

Monitoring

RBAC

Security policies

Troubleshooting

Training

Cost optimization

Cross-functional collaboration

Agile/DevOps

Tools

Databricks

AWS

Terraform

Job description

Title: Databricks platform administrator

Responsibilities:

Below are the general responsibilities.

• Databricks platform (not as a data engineer, but as a Databricks platform administrator)
• Cloud experience
• Hadoop/Databricks Operations is part of the Hadoop Support team. This team is responsible for managing the Hadoop/Databricks plant across the system.
• Working within an Agile delivery (Scrum/Kanban/Scrumban)/ DevOps methodology. Support the application development teams and help them in debugging and fixing an issue.
• Configure, deploy and maintain Databricks clusters and workspaces.
• Use Terraform to provision and deploy Databricks clusters, workspaces, and associated AWS resources.
• Ensure consistency and repeatability by treating infrastructure as code
• Monitor cluster performance, resource utilization, and troubleshoot issues to ensure optimal performance.
• Implement and manage access controls and security policies to protect sensitive data.
• Manage platform feature and software releases and updates.
• Provide technical support to Databricks users, including troubleshooting and issue resolution.
• Conduct training sessions to educate users on platform best practices and capabilities.
• Optimize cluster configurations for performance and cost-effectiveness.
• Monitor and manage auto-scaling to ensure resources are allocated efficiently.
• Implement and enforce security policies, RBAC, access controls, and encryption mechanisms.
• Develop and maintain backup and disaster recovery strategies to ensure data integrity and availability.
• Collaborate with data engineers to integrate Databricks with other data sources, data warehouses, and data lakes.
• Monitor and manage platform costs, including optimizing resource utilization to control expenses.
• Collaborate with cross-functional teams, including data scientists, data engineers, and business analysts to understand their requirements and provide technical solutions