Databricks Platform Engineer / Databricks Administrator

Blue Cloud Softech Solutions Limited; Bengaluru Urban; On-site; INR 1,800,000 - 2,400,000; Full time

Job summary

Blue Cloud Softech Solutions Limited is seeking a Databricks Platform Engineer/Administrator in Bengaluru on a US shift (6:00 PM to 3:00 AM). The role covers managing Databricks infra on AWS, building CI/CD pipelines with Jenkins, and implementing IaC with Terraform.

The ideal candidate has 4+ years in DevOps, solid Python/Groovy scripting, and experience with Unity Catalog, clusters, and security in Databricks.

Qualifications

• Bachelor's degree in CS/IT/Engineering or related field.
• 4+ years in DevOps, cloud engineering or related roles.
• Experience with Databricks platform and AWS services.
• Hands-on with Jenkins/CI-CD and Git version control.
• Experience with Databricks Unity Catalog, clusters and security.
• Familiar with monitoring/logging tools like CloudWatch, DataDog.
• Strong Python and Groovy scripting skills.
• Able to work in Agile/Scrum environments.

Responsibilities

• Manage and maintain AWS infrastructure for scalability, security and cost.
• Design, build, and manage Databricks infra on AWS.
• Support deployment of Databricks-based data platforms and job orchestration.
• Implement IaC using Terraform or CloudFormation.
• Develop automation scripts in Python and Shell.
• Build CI/CD pipelines using Jenkins/CloudBees for deployments.
• Collaborate with cross-functional teams to enable reliable delivery.
• Monitor system performance and reliability; troubleshoot issues.
• Perform root cause analysis with preventive actions.

Skills

Databricks

Python scripting

AWS cloud engineering

CI/CD pipelines

Jenkins CloudBees

Groovy

Agile/Scrum

Shell scripting

Education

Bachelor's degree in Computer Science/IT/Engineering

Tools

Terraform

AWS CloudFormation

CloudWatch

DataDog

Job description

Job Title: Databricks Platform Engineer / Databricks Administrator

Location: [Bangalore / US Shift hours (6.00 pm to 3.00 am]

Function: AWS | Databricks | Jenkins | Terraform | Python

We are seeking a highly skilled, hands-on engineer with deep expertise in Databricks, CI/CD, and cloud infrastructure on AWS. The ideal candidate should be well-versed in Databricks infrastructure management, automation, and deployment pipelines, with strong programming skills in Python.

Key Responsibilities

• Manage and maintain AWS cloud infrastructure, ensuring scalability, security, and cost efficiency
• Design, build, and manage Databricks infrastructure on AWS
• Support deployment and operationalization of Databricks-based data platforms, including cluster management and job orchestration
• Implement Infrastructure as Code (IaC) practices using tools such as Terraform or AWS CloudFormation
• Develop automation scripts and tools using Python and Shell Scripting to streamline operational processes
• Design, implement, and optimize CI/CD pipelines using CloudBees (Jenkins) to support application and data platform deployments
• Collaborate with software engineering, data engineering, and architecture teams to enable reliable and efficient delivery pipelines
• Monitor system performance, availability, and reliability using enterprise monitoring solutions
• Troubleshoot production issues and implement root cause analysis with preventive action.

Required Qualifications

• Bachelor s degree in Computer Science, Information Technology, Engineering, or related discipline
• 4+ years of experience in DevOps, cloud engineering, or related roles
• Exposure to Databricks platform (cluster setup, job execution, notebooks)
• Hands-on experience with AWS services (EC2, S3, IAM, VPC, Lambda, etc.)
• Experience with CloudBees/Jenkins for CI/CD pipeline implementation
• Experience with Git-based version control systems
• Experience in Data bricks Platform Administration - Unity Catalog, Clusters, Platform security and access management.
• Hands on Databricks Apps and Databricks Genie Admin / Configuration.
• Experience with monitoring and logging tools such as CloudWatch, DataDog
• Understanding of cloud security, networking, and system architecture principles
• Strong scripting/programming skills in Python and Groovy
• Experience working in Agile/Scrum environments