Senior Data Engineer (GCP)

EPAM Systems; Malaysia; On-site; MYR 180,000 - 240,000; Full time

Benefits offered by this job

Mentoring programs

Certification support

Online courses

Global collaboration

Job summary

EPAM Systems is seeking a Senior Data Engineer (Google Cloud Platform) to design and build reliable ETL/ELT pipelines, optimize SQL, and orchestrate workloads with Airflow.

You will partner with clients and delivery teams to improve quality, performance and operational stability across cloud data platforms, working across GCP and AWS with CI/CD practices and data modeling excellence.

Qualifications

• Hands-on experience building ETL and ELT pipelines and data warehousing.
• Advanced SQL for transformation, validation and performance tuning.
• Proficient in Python for pipeline development and software engineering practices.
• Experience with PySpark or Apache Beam for distributed processing.
• Cloud experience with Google Cloud Platform and AWS for data workloads.
• Workflow orchestration background with Airflow or Cloud Composer.
• Knowledge of data modeling, data lifecycle management and data quality controls.
• Clear communication to explain trade-offs and risks to stakeholders.

Responsibilities

• Design and build scalable ETL and ELT pipelines using Python, SQL, BigQuery and cloud-native services.
• Develop and optimize complex SQL for transformation, validation, troubleshooting and performance.
• Create and maintain data models and warehousing solutions that support quality, availability and scale.
• Orchestrate workflows in Apache Airflow, including scheduling, monitoring and incident resolution.
• Support pipeline delivery and platform operations across Google Cloud Platform and AWS.
• Improve CI/CD, source control and automated deployment practices for data solution.
• Investigate data, integration and infrastructure issues, driving root-cause analysis and fixes.
• Collaborate with clients and cross-functional teams to clarify requirements, risks, dependencies and delivery plans.

Skills

ETL/ELT pipelines

Advanced SQL

Python

PySpark/Beam

GCP

Airflow

Data modeling

Cloud data platforms

CI/CD practices

Communication

Education

Bachelor's degree in Computer Science/Engineering

Tools

BigQuery

Google Cloud Composer

Terraform

Kubernetes

Job description

We are seeking a Senior Data Engineer (Google Cloud Platform). You will build reliable ETL and ELT pipelines, optimize SQL and orchestration in Apache Airflow and help teams turn raw data into trusted, ready-to-use datasets. You will partner with clients and delivery teams to improve quality, performance and operational stability across cloud data platforms.

Responsibilities

• Design and build scalable ETL and ELT pipelines using Python, SQL,Big Query and cloud-native services
• Develop and optimize complex SQL for transformation, validation, troubleshooting and performance
• Create and maintain data models and warehousing solutions that support quality, availability and scale
• Orchestrate workflows in Apache Airflow, including scheduling, monitoring and incident resolution
• Support pipeline delivery and platform operations across Google Cloud Platform and Amazon Web Services (AWS)
• Improve CI/CD, source control and automated deployment practices for data solution
• Investigate data, integration and infrastructure issues, driving root-cause analysis and fixes
• Collaborate with clients and cross-functional teams to clarify requirements, risks, dependencies and delivery plans

Requirements

• Hands-on experience building ETL and ELT pipelines and supporting data warehousing
• Advanced SQL skill set including optimization, data validation and complex transformation
• Python proficiency for pipeline development and software engineering practices
• Experience with distributed processing using PySpark or Apache Beam
• Practical cloud experience with Google Cloud Platform and or Amazon Web Services (AWS) for data workloads
• Workflow orchestration background with Apache Airflow or Google Cloud Composer
• Knowledge of data modeling data lifecycle management and data quality controls
• Clear communication with the ability to explain trade-offs, risks and solution options to stakeholders

Nice to have

• Infrastructure-as-code exposure using Terraform and or Kubernetes
• Streaming familiarity with Kafka and event-driven patterns
• Data integration tooling experience such as Fivetran

We offer

• By choosing EPAM, you're getting a job at one of the most loved workplaces according to Newsweek 2021 & 2022&2023.
• Employee ideas are the main driver of our business. We have a very supportive environment where your voice matters
• You will be challenged while working side-by-side with the best talent globally. We work with top-notch technologies, constantly seeking new industry trends and best practices
• We offer a transparent career path and an individual roadmap to engineer your future & accelerate your journey
• At EPAM, you can find vast opportunities for self-development: online courses and libraries, mentoring programs, partial grants of certification, and experience exchange with colleagues around the world. You will learn, contribute, and grow with us