Data Engineer (Azure, Databrick)

Recruit Express Pte Ltd; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Recruit Express Pte Ltd is seeking an experienced Data Engineer to design, build and optimise data pipelines and platforms in our Singapore team. You will work across data management, AI/ML enablement and governance to deliver scalable solutions.

The role requires hands-on expertise with Azure, Databricks, Python, PySpark and SQL, plus CI/CD and data quality practices. Join us to enable data-driven decision making and accelerate analytics initiatives.

Qualifications

• 5+ years of experience in data management or data engineering.
• Hands-on experience designing and implementing data architectures using Azure and Databricks.
• Strong programming skills in Python, PySpark and SQL.

Responsibilities

• Design, develop and maintain data pipelines that extract data from business applications, databases and external systems.
• Implement data quality checks and validation processes to ensure data accuracy, consistency and completeness.
• Perform data cleansing, aggregation, filtering, transformation and enrichment.
• Establish and maintain governance frameworks for data and algorithms used in analytics, automated decision-making and analytical applications.
• Develop and maintain logical and physical data models, including data structures, relationships and constraints.
• Ensure data solutions comply with security, governance and organisational best practices.

Skills

Data pipelines

Python

PySpark

SQL

Azure

Databricks

CI/CD

Data quality

Data governance

Tools

Azure DevOps

Databricks Unity Catalog

Microsoft Purview

Databricks LakeFlow Jobs

Job description

Key Responsibilities

Data Management & Transformation

• Design, develop and maintain data pipelines that extract data from business applications, databases and external systems.
• Implement data quality checks and validation processes to ensure data accuracy, consistency and completeness.
• Perform data cleansing, aggregation, filtering, transformation and enrichment.
• Establish and maintain governance frameworks for data and algorithms used in analytics, automated decision-making and analytical applications.
• Develop and maintain logical and physical data models, including data structures, relationships and constraints.
• Ensure data solutions comply with security, governance and organisational best practices.

Data Platform Optimisation & Automation

• Develop and maintain CI/CD pipelines for data deployments and cloud resource provisioning.
• Optimise data pipelines and processing workflows for performance, scalability and efficiency.
• Optimise data models and algorithms for data quality, security, governance, performance and scalability.
• Monitor data warehouse and ETL platform capacity, including capacity planning and forecasting.
• Monitor and tune data platforms, troubleshoot performance bottlenecks and implement optimisation techniques such as caching and indexing.
• Monitor platform resource and credit consumption and perform regular housekeeping activities.
• Support the deployment, integration and maintenance of AI/ML solutions within the data platform.

Collaboration & Business Enablement

• Partner with the Data Lead and business users to manage and maximise the value of data as a business asset.
• Work with stakeholders to understand data requirements and translate them into scalable technical solutions.
• Guide business users in the creation and maintenance of reports and dashboards.
• Collaborate with technology, data and business teams to improve data quality, accessibility and usability.
• Any other ad-hoc duties as assigned by supervisor

Requirements

• 5+ years of experience in data management or data engineering, covering areas such as data integration, data modelling, data quality and platform optimisation.
• Hands-on experience designing and implementing data architectures using Azure and Databricks.
• Strong programming skills in Python, PySpark and SQL.
• Strong experience with relational and non-relational databases, including SQL and NoSQL databases.
• Hands-on experience with Azure DevOps and Databricks LakeFlow Jobs, including version control, CI/CD and pipeline deployment.
• Experience with data catalogue and governance tools such as Databricks Unity Catalog and Microsoft Purview.
• Experience supporting AI/ML model deployment, feature engineering and production inference is an advantage.