Senior Data Engineer - DT&D Transformation & Automation

BOC Aviation; Singapore; On-site; SGD 150,000 - 190,000; Full time

Job summary

BOC Aviation is seeking a Senior Data Engineer to design, develop and optimize data pipelines across diverse sources in a cloud environment. You will implement data quality checks, govern data and algorithms, and manage complex data models.

The role requires 10+ years in data management with strong Python/PySpark, SQL, and Azure Databricks experience, along with proficiency in CI/CD practices and BI tools.

Qualifications

• Bachelor’s degree or higher in a quantitative field.
• Ten years in data management disciplines or related areas.

Responsibilities

• Design and develop new data pipelines and manage existing ones from various sources.
• Implement data quality checks to ensure data accuracy, consistency and completeness.
• Transform data with cleansing, aggregation, filtering and enrichment techniques.
• Establish governance of data and algorithms used for analysis and ML deployment.
• Manage logical and physical data models.
• Ensure compliance with security and governance best practices.
• Implement and maintain CI/CD pipelines and cloud provisioning.
• Optimize data pipelines for performance and scalability.
• Support deployment of AI solutions within the data platform.
• Guide business users to create and maintain reports and dashboards.

Skills

Python/PySpark

SQL

Azure

Databricks

CI/CD (Azure DevOps)

Data governance

Power BI

Big data pipelines

NoSQL

Education

Bachelor’s degree in computer science, data science, software engineering, information systems, or related quantitative field

Master’s degree preferred

Tools

Azure cloud services

Databricks LakeFlow

Unity Catalog

Microsoft Purview

Power BI

Job description

BOC Aviation is seeking a Senior Data Engineer to design, develop and optimize data pipelines across diverse sources in a cloud environment. You will implement data quality checks, govern data and algorithms, and manage complex data models.

The role requires 10+ years in data management with strong Python/PySpark, SQL, and Azure Databricks experience, along with proficiency in CI/CD practices and BI tools.