Data Engineer Big Data (Databricks)

OPTIMUM SOLUTIONS (SINGAPORE) PTE LTD; Singapore; On-site; SGD 70,000 - 100,000; Full time

Job summary

A leading digital solutions provider in Singapore is looking for a Data Engineer with over 5 years of experience in data engineering and strong programming skills in Python or Java. The role involves guiding data engineering processes, designing data pipelines, and ensuring data quality in alignment with national standards. Candidates must have knowledge of big data frameworks and experience with cloud platforms like AWS or Azure. This position also includes mentoring the data engineering team and collaborating cross-functionally.

Qualifications

• 5+ years of experience in data engineering, cloud infrastructure, or similar roles.
• Experience with cloud platforms such as AWS, Azure, or GCP.
• Strong programming skills in Python, Java, or Scala.

Responsibilities

• Guide and plan data engineering processes and architecture.
• Design, develop, and maintain data pipelines and lakes.
• Ensure data quality and compliance with security requirements.

Skills

Big data frameworks (e.g., Spark)

Python

Java

SQL

Data governance and compliance

Education

Bachelor’s degree in Computer Science or related field

Tools

Databricks

AWS

Azure

Power BI

AWS QuickSight

Job description

Responsibilities

• Guide and plan data engineering processes for digital systems and products, providing technical leadership and shaping strategy, architecture, and implementation aligned with national digital government standards.
• Lead technical discussions with data engineering teams and stakeholders, establishing data architecture standards, DevOps best practices, and ensuring consistency, quality, and system reliability across initiatives.
• Design, develop, and maintain data pipelines, databases, and data lakes to collect, process, harmonize, and store data from multiple platforms, supporting analytical, reporting, and AI workloads.
• Implement data validation, cleansing, and monitoring processes to ensure data quality, integrity, and compliance with governance and security requirements.
• Collaborate with product teams, analysts, and stakeholders to translate business requirements into technical specifications, scalable solutions, and data visualizations for reporting.
• Mentor and develop the data engineering team’s capabilities, promote knowledge sharing, and contribute to process improvements and adoption of emerging technologies.
• Maintain documentation of data systems, configurations, and engineering processes, and troubleshoot issues to ensure reliable system performance.

Requirements

• Familiarity with big data frameworks (e.g., Spark), Databricks, version control systems (e.g., Git), and data governance, security, and compliance best practices.
• Bachelor’s degree in Computer Science, Data Science, Engineering, Information Technology, or a related technical field.
• 5+ years of experience in data engineering, cloud infrastructure, platform engineering, or similar technical roles, including experience with cloud platforms such as AWS, Azure, or GCP.
• Strong programming skills in Python, Java, or Scala, and solid knowledge of SQL, relational and non-relational databases, data modelling, and ETL/ELT processes.
• Experience designing, developing, and maintaining data pipelines, data lakes, and databases to support analytics, reporting, and AI workloads.
• Experience with business intelligence and data visualization tools such as Power BI or AWS QuickSight.
• Certifications such as AWS Certified Data Engineer or Databricks Certified Data Engineer will be good to have.