Data Engineer: Databricks, AWS Data Pipelines

sggovterp; Singapore; On-site; SGD 60,000 - 90,000; Full time

Job summary

Singapore Global Network (SGN) at the Economic Development Board is seeking a Data Engineer to design and maintain robust data pipelines using Databricks and AWS services. You will build a medallion architecture, implement data governance with Unity Catalog, and prepare analytics-ready datasets for reporting and insights.

The role emphasizes end-to-end data solutions, ETL/ELT patterns, and collaboration with stakeholders to enable advanced analytics and BI across global networks.

Qualifications

• 2-5 years of data engineering experience with ability to work independently on data pipeline projects.
• Diploma or degree in Computer Science, Engineering, IT, Data Science or related disciplines.
• Proficiency with Databricks platform including workflows, notebooks, and cluster management.
• Experience implementing medallion architecture for data lake solutions.
• Hands-on experience with Unity Catalog for data governance and cataloguing.
• Strong Python and SQL skills for data manipulation and transformation.
• Experience with Jupyter notebooks and collaborative data science environments.
• Working knowledge of AWS data services (S3, Athena, RDS).
• Understanding of ETL/ELT principles and data pipeline design patterns.
• Basic knowledge of machine learning concepts and ML pipeline implementation.
• Singaporean or PR

Responsibilities

• Design and implement data pipelines using Databricks workflows and notebooks.
• Build and maintain medallion architecture (Bronze, Silver, Gold layers) for data processing and transformation.
• Develop ETL processes to integrate data from various SaaS platforms and internal systems.
• Implement data governance practices using Unity Catalog for data discovery, lineage, and access control.
• Create and maintain Jupyter notebooks for data exploration and collaborative analytics.
• Design scalable data storage solutions using AWS services (S3, RDS, Athena).
• Build data models and prepare datasets optimized for reporting and business intelligence.
• Implement ML pipelines for basic predictive analytics and data insights.
• Ensure data quality, validation, and monitoring across all data workflows.
• Collaborate with stakeholders to understand data requirements and deliver analytics-ready datasets.

Skills

Databricks

Python

SQL

Jupyter

AWS

Unity Catalog

Medallion architecture

ETL/ELT

Data governance

Data modeling

Education

Diploma or degree in Computer Science or related field

Tools

Databricks

AWS

Power BI

Job description

Singapore Global Network (SGN) at the Economic Development Board is seeking a Data Engineer to design and maintain robust data pipelines using Databricks and AWS services. You will build a medallion architecture, implement data governance with Unity Catalog, and prepare analytics-ready datasets for reporting and insights.

The role emphasizes end-to-end data solutions, ETL/ELT patterns, and collaboration with stakeholders to enable advanced analytics and BI across global networks.