Senior Data Engineer ( DevOps)

Dexian Asia Pacific; Kuala Lumpur; On-site; MYR 180,000 - 240,000; Full time

Job summary

Dexian Asia Pacific is seeking an experienced back-end developer with data analytics expertise in Kuala Lumpur. The role focuses on building and supporting upstream subsurface data management apps, including ingestion pipelines and workflows.

The candidate will migrate data from legacy systems to modern data warehouses, ensure data integrity, and optimize workflows. OSdu community contributions and open-source platform work are encouraged.

Qualifications

• Degree in Computer Science, Engineering, or Data Engineering.
• Proficiency in Python and SQL scripting, experience with Oracle databases, APIs and Postman.
• Familiarity with DevOps tools (GitHub/GitLab, CI/CD) and cloud infrastructure (Azure, OpenShift, Airflow).
• Data engineering experience with Snowflake and Azure Data Factory is a plus.

Responsibilities

• Develop and support Upstream Subsurface data management applications, including ingestion pipelines and workflows.
• Migrate data from legacy systems to modern data warehouses, ensuring data integrity and optimizing workflows throughout the process.
• Contribute to the Open Subsurface Data Universe (OSDU) community by building open-source data platforms and defining industry data standards.
• Advocate and model secure coding practices.
• Promote quality development principles and techniques.
• Design and implement data pipelines for application and infrastructure deployment.
• Analyze gaps between legacy and target systems, identify missing data, and recommend improvements for completeness.
• Mentor others through teaching, coaching, and providing feedback.
• Develop deep expertise in relevant technology stacks through diverse development experiences.

Skills

Python

SQL

CI/CD

Secure coding practices

Communication

Mentoring

Growth mindset

Data analytics

Power BI

Education

Bachelor's degree in Computer Science, Engineering, or Data Engineering

Tools

GitHub/GitLab CI/CD

Azure

Terraform

OpenShift

Airflow

Snowflake

Azure Data Factory

Postman

Job description

Experienced professional with strong expertise in back-end development and data analytics. Proficient in Python, and SQL, with hands-on experience in cloud-native development on Azure and Infrastructure as Code (IaC) using Terraform Enterprise.

Well-versed in designing and implementing CI/CD pipelines to automate deployment of both infrastructure and application code.

Role Overview:

• Develop and support Upstream Subsurface data management applications, including ingestion pipelines and workflows.
• Migrate data from legacy systems to modern data warehouses, ensuring data integrity and optimizing workflows throughout the process.
• Contribute to the Open Subsurface Data Universe (OSDU) community by building open-source data platforms and defining industry data standards.

Key Responsibilities

• Advocate and model secure coding practices.
• Promote quality development principles and techniques.
• Design and implement data pipelines for application and infrastructure deployment.
• Analyze gaps between legacy and target systems, identify missing data, and recommend improvements for completeness.
• Mentor others through teaching, coaching, and providing feedback.
• Develop deep expertise in relevant technology stacks through diverse development experiences.

Requirements:

• Degree in Computer Science, Engineering, or Data Engineering.
• Proficiency in Python and SQL scripting, experience with Oracle databases, APIs and Postman.
• Familiarity with DevOps tools (GitHub/GitLab, CI/CD) and cloud infrastructure (Azure, OpenShift, Airflow).
• Data engineering experience with Snowflake and Azure Data Factory is a plus.
• Strong communication skills for collaborating with team members, stakeholders, and external partners.
• Growth mindset with a proven track record of self-learning and continuous skill development.
• Experience with Power BI and Power Apps is an advantage.