Data Engineer- Databricks

RIDIK Pte Ltd; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

RIDIK Pte Ltd invites a senior data engineer to join a 1-year contract at Tanjong Pagar, Singapore. You will design and maintain production data pipelines using PySpark and Databricks, optimize Spark jobs, and collaborate with cross-functional teams.

The ideal candidate has 5+ years in data engineering, 2–3 years hands-on with Databricks, strong Python skills, and experience with data modelling (dimensional, data vault or lakehouse) and Delta Lake.

Qualifications

• 5+ years in data engineering or related roles.
• 2–3 years hands-on Databricks experience.
• Strong Python programming for data processing and automation.
• Experience building production data pipelines at scale.
• Experience with agile development environments.

Responsibilities

• Build and maintain production data pipelines at scale.
• Design data models and optimization strategies.
• Implement data governance and security best practices.
• Develop and optimize PySpark data transformations.
• Collaborate with cross-functional teams to deliver data solutions.

Skills

PySpark

Databricks Platform

Data Engineering

Data Modelling

Delta Lake

Python

SQL

Cloud Platforms

Structured Streaming

CI/CD

Tools

Databricks Platform

Spark SQL

Job description

Overview

Duration: 1 year contract subject to renewal
Location: Tanjong Pagar

Job Details

• Data Engineering: Strong foundation in data engineering principles, ETL/ELT processes, and data pipeline design patterns
• PySpark: Proven hands-on experience developing data pipelines using PySpark, including DataFrames API, Spark SQL, and performance optimization
• Databricks Platform: Practical experience with Databricks workspace, cluster management, notebooks, and job orchestration
• Workspace AI Agent: Knowledge of Databricks Workspace AI Agent capabilities and integration
• Data Modelling: Experience implementing data models including dimensional modeling, data vault, or lakehouse architectures
• Delta Lake: Understanding of Delta Lake features including ACID transactions, schema evolution, and optimization techniques
• Python: Strong Python programming skills for data processing and automation

Additional Skills

• SQL proficiency for data querying and transformation
• Experience with cloud platforms (Azure, AWS, or GCP)
• Understanding of data governance and security best practices
• Knowledge of streaming data processing (Structured Streaming)
• Familiarity with DevOps practices and CI/CD pipelines
• Experience with version control systems (Git)
• Understanding of data quality frameworks and testing methodologies

Professional Experience

• Minimum 5 years in data engineering or related roles
• At least 2-3 years of hands-on experience with Databricks platform
• Proven track record of refactoring legacy code to modern frameworks
• Experience building and maintaining production data pipelines at scale
• Background working across multiple data sources and formats
• Experience in agile development environments
• Nice to have Databricks Certified Data Engineer Associate OR Databricks Certified Data Engineer Professional

Additional Certifications (Preferred)

• Databricks Certified Associate Developer for Apache Spark
• Cloud platform certifications (Azure Data Engineer Associate, AWS Certified Data Analytics, or Google Cloud Professional Data Engineer)
• Relevant data engineering or big data certifications