Data Engineer (MY)

Second Talent; Kuala Lumpur; On-site; MYR 90,000 - 150,000; Full time

Job summary

Second Talent is seeking an experienced Data Engineer to join the technology team in Kuala Lumpur. You will build and manage scalable batch and streaming data pipelines on GCP, enabling advanced analytics and AI solutions.

You will work with BigQuery, Dataflow, Dataproc, Composer, Dataform, and Cloud Functions, loading and transforming data from diverse sources, and guiding migration from legacy systems. Collaboration with analysts and business teams is essential to deliver reliable,

Qualifications

• 3–5 years of professional experience in data engineering or ETL development.
• Hands-on experience with GCP Data Stack (BigQuery, Dataflow, Composer, Dataproc).
• Strong SQL and Python programming skills.
• Understanding of data modelling and performance optimization techniques.
• Willingness to learn and contribute to large-scale migration projects.
• Familiarity with Azure Data Stack is a plus.

Responsibilities

• Architect, develop, and maintain scalable batch and streaming data pipelines on GCP (BigQuery, Dataflow, Dataproc, Composer, Dataform, Cloud Functions).
• Design and execute processes to load, transform, and optimize data within BigQuery for analytics and reporting.
• Integrate data from diverse sources, including external APIs and databases.
• Lead data migration initiatives from Oracle and MicroStrategy to the cloud.
• Enforce data quality, governance, and security across platforms.
• Collaborate with analysts and business teams to support reporting requirements.

Skills

Data engineering

ETL development

SQL

Python

Data modelling

Performance optimization

Data governance

Tools

BigQuery

Dataflow

Dataproc

Composer

Dataform

Cloud Functions

Job description

About the Role
We areseeking an experiencedData Engineerto join the technology team. This role is pivotal in building and managing the robustdata pipelinesand platforms that are the foundation for our advanced analytics andAI solutions.

Key Responsibilities

• Architect, develop, and maintain scalable batch andstreaming data pipelinesutilizing theGoogle Cloud Platform (GCP),includingBigQuery, Dataflow, Dataproc, Composer, Dataform,andCloud Functions.
• Design and execute processes to load, transform, andoptimize data within BigQueryto ensure high performance for analytics and reporting.
• Integrate data from diverse and complex sources, including external APIs, various databases, and flat files.
• Play a key role indata migrationinitiatives, assisting in the transition from legacy systems such asOracleandMicroStrategyto the new cloud environment.
• Implement and enforce best practices fordata quality, data governance, and security compliance across all platforms.
• Collaborate effectively with data analysts and business teams to understand requirements and provide the necessary data support for reporting.

Requirements

• A minimum of3-5 yearsof professional experience indata engineeringor complexETL development.
• Deep, hands-on experience with theGCP Data Stack(must have:BigQuery, Dataflow, Composer, Dataproc).
• Solid, practical programming skills in bothSQLandPython.
• A strong understanding ofdata modellingconcepts and databaseperformance optimizationtechniques.
• A proactive and curious mindset, with a strong willingness to learn and contribute tolarge-scale migration projects.
• Familiarity or prior experience with theAzure Data Stackis a significant plus.