Databricks Infrastructure Engineer

APAR TECHNOLOGIES PTE. LTD.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

APAR TECHNOLOGIES PTE. LTD. in Singapore seeks an experienced Databricks Infrastructure Engineer to design, deploy and optimise Databricks workspaces and clusters on AWS, ensuring performance and governance.

You will automate infrastructure with Terraform or Databricks Asset Bundles (DAB), build CI/CD pipelines with GitLab, and apply security, data protection and compliance controls while collaborating with Data Engineers and Security teams.

Qualifications

• 3+ years of Databricks platform administration or related engineering roles.
• Hands-on Databricks administration including workspace and cluster management.
• Experience with AWS services (S3, IAM, VPC, security groups).
• Experience with Terraform or Databricks Asset Bundles (DAB).
• Knowledge of security and compliance practices for cloud environments.

Responsibilities

• Design, deploy and maintain Databricks workspaces and clusters on AWS.
• Monitor and optimise Databricks platform performance and resource usage.
• Configure AWS services for Databricks environments (S3, IAM, VPC, security groups).
• Manage infrastructure using Terraform or DAB.

Skills

Databricks platform administration

AWS cloud services

Scripting (Python Bash PowerShell)

Security & compliance

CI/CD understanding

Stakeholder communication

Tools

Terraform

Databricks Asset Bundles (DAB)

GitLab

SonarQube

Fortify

Nexus IQ

PowerShell

Job description

We are looking for an experienced Databricks Infrastructure Engineer to manage, maintain and optimise Databricks infrastructure hosted on AWS. The role will focus on Databricks platform administration, infrastructure automation, security, governance and operational support.

Key Responsibilities

Databricks & Infrastructure Management

• Design, deploy and maintain Databricks workspaces and clusters on AWS.
• Monitor and optimise Databricks platform performance and resource utilisation.
• Configure AWS services such as S3, IAM, VPC and security groups for Databricks environments.
• Manage infrastructure using Terraform or Databricks Asset Bundles (DAB).
• Support multiple datamarts and data processing environments.

CI/CD & Automation

• Develop and maintain CI/CD pipelines using GitLab and approved tools.
• Support automated deployment workflows for Databricks components.
• Implement code quality and security checks using tools such as SonarQube, Fortify and Nexus IQ.
• Develop scripts and automation using Python, Bash or PowerShell.

Security & Compliance

• Implement security and governance controls for Databricks environments.
• Support data classification, data protection and data retention requirements.
• Manage secrets, encryption mechanisms and key rotation.
• Support vulnerability assessment, remediation and security scanning activities.
• Assist with security monitoring, incident response and compliance activities.
• Support security testing and acceptance activities.

Platform Operations & Support

• Monitor system health, performance and operational issues.
• Configure notifications for data processing status and system errors.
• Support access management and user onboarding.
• Assist with training environment setup and maintenance.
• Provide technical guidance for data pipelines and platform-related issues.

Collaboration

• Work closely with Data Engineers, Data Analysts and Security teams.
• Collaborate with government stakeholders and technical teams.
• Support the implementation of secure and reliable data platforms.

Key Requirements

• Minimum 3 years of experience in Databricks platform administration, infrastructure or related engineering roles.
• Strong hands‑on experience with Databricks, including workspace and cluster management.
• Experience working with AWS, particularly services such as S3, IAM, VPC and security groups.
• Experience with Terraform or Databricks Asset Bundles (DAB).
• Experience with GitLab and CI/CD pipelines is an advantage.
• Knowledge of security and compliance practices for cloud environments.
• Experience with containerisation and orchestration technologies is an advantage.
• Strong scripting skills in Python, Bash or PowerShell.
• Knowledge of security scanning and vulnerability management tools such as SonarQube, Fortify and Nexus IQ is an advantage.
• Understanding of encryption, secrets management and key management practices.
• Strong troubleshooting, communication and stakeholder management skills.

Preferred Certifications

• Databricks Certified Associate or higher.
• AWS Certified DevOps Engineer or relevant AWS certification.

EA Number : 11C4879