Senior Data Engineer (Cloud)

EPAM Systems; Kuala Lumpur; On-site; MYR 180,000 - 300,000; Full time

Benefits offered by this job

Newsweek recognition

Mentoring program

Online courses

Certification grants

Job summary

EPAM Systems in Kuala Lumpur is seeking a Senior Data Engineer to design, build and run reliable ETL and ELT pipelines for enterprise data platforms using Python, SQL and Apache Airflow across Azure, GCP or AWS. You will troubleshoot issues, improve delivery automation and partner with clients on requirements and trade-offs.

The role emphasizes scalable pipelines, complex SQL, data modeling and warehousing, Airflow orchestration, and CI/CD improvements, while collaborating with cross‑functional

Qualifications

• Hands-on experience building ETL/ELT pipelines and data warehousing.
• Advanced SQL skills including optimization and data validation.
• Python proficiency for pipeline development.
• Experience with distributed processing using PySpark or Apache Beam.
• Cloud experience with Azure, GCP or AWS for data workloads.
• Workflow orchestration with Airflow or Cloud Composer.
• Knowledge of data modeling and data quality control.
• Clear communication of trade-offs, risks and options.

Responsibilities

• Design and build scalable ETL/ELT pipelines using Python, SQL and cloud-native services.
• Write, optimize and troubleshoot complex SQL for transformations, validation and performance.
• Create and maintain data models and warehouse structures that support quality, availability and scale.
• Orchestrate workflows in Apache Airflow including scheduling, monitoring and incident resolution.
• Support pipeline delivery and platform operations across Azure, GCP or AWS.
• Improve CI/CD, source control and automated deployment for data solutions.
• Investigate data, integration and infrastructure issues; drive root-cause analysis.
• Collaborate with clients and cross-functional teams to align on requirements and delivery plans.

Skills

ETL pipelines

SQL

Python

PySpark

Apache Beam

Airflow

Cloud platforms

Data modeling

Data quality

Communication

Tools

Apache Airflow

PySpark

Apache Beam

Azure

GCP

AWS

Job description

We are seeking a Senior Data Engineer to design, build and run reliable ETL and ELT pipelines for enterprise data platforms. You will use Python, SQL and Apache Airflow with cloud services across Microsoft Azure, Google Cloud Platform (GCP) or Amazon Web Services (AWS). You will troubleshoot issues, improve delivery automation and partner with clients on requirements and trade-offs.

Responsibilities

• Design and build scalable ETL and ELT pipelines using Python, SQL and cloud-native services
• Write, optimize and troubleshoot complex SQL for transformations, validation and performance
• Create and maintain data models and warehouse structures that support quality, availability and scale
• Orchestrate workflows in Apache Airflow including scheduling, monitoring and incident resolution
• Support pipeline delivery and platform operations across Azure, Google Cloud Platform (GCP) or Amazon Web Services (AWS)
• Improve continuous integration and continuous delivery (CI/CD), source control and automated deployment for data solutions
• Investigate data, integration and infrastructure issues, drive root-cause analysis and implement fixes
• Collaborate with clients and cross-functional teams to align on requirements, risks, dependencies and delivery plans

Requirements

• Hands-on experience building ETL and ELT pipelines and supporting data warehousing
• Advanced SQL skill sets including optimization, data validation and complex transformations
• Python proficiency for pipeline development and software engineering practices
• Experience with distributed processing using PySpark or Apache Beam
• Practical cloud experience with Microsoft Azure, Google Cloud Platform (GCP) or Amazon Web Services (AWS) for data workload
• Workflow orchestration background with Apache Airflow or Google Cloud Composer
• Knowledge of data modeling, data lifecycle management and data quality control
• Clear communication with the ability to explain trade-offs, risks and solution options to stakeholders

Nice to have

• Infrastructure as code exposure using Terraform or Kubernetes
• Streaming familiarity with Kafka and event-driven patterns
• Data integration tooling experience with Fivetran

We offer

• By choosing EPAM, you're getting a job at one of the most loved workplaces according to Newsweek 2021 & 2022&2023.
• Employee ideas are the main driver of our business. We have a very supportive environment where your voice matters
• You will be challenged while working side-by-side with the best talent globally. We work with top-notch technologies, constantly seeking new industry trends and best practices
• We offer a transparent career path and an individual roadmap to engineer your future & accelerate your journey
• At EPAM, you can find vast opportunities for self-development: online courses and libraries, mentoring programs, partial grants of certification, and experience exchange with colleagues around the world. You will learn, contribute, and grow with us