Data Engineer (Spark, Python, AWS/GCP, Databricks)

Epam Systems; Hyderabad; Presencial; INR 1.200.000 - 2.400.000; Jornada completa

Descripción de la vacante

Epam Systems in Hyderabad, India is seeking a Data Engineer with 6+ years of experience to design, build and optimize scalable data pipelines using Spark and Python. You will collaborate across teams to deliver data-driven solutions that perform reliably at scale.

The role requires hands-on AWS or GCP plus Databricks, Delta Lake, and strong understanding of data warehousing and Big Data concepts. Cloud-based ETL/ELT development and performance tuning are core expectations.

Formación

• Strong experience in Python and PySpark

Responsabilidades

• Develop scalable data pipelines using Spark and Python
• Design and implement cloud-based data solutions on AWS or GCP
• Build, optimize, and maintain ETL processes
• Collaborate with cross-functional teams to deliver data-driven solutions
• Ensure data quality, performance, and reliability across platforms

Conocimientos

Python

PySpark

SQL

ETL/ELT pipelines

Data Warehousing

Big Data concepts

Spark performance tuning

Delta Lake

Databricks

Databricks Workflows

Notebooks

Unity Catalog

AWS

GCP

S3

EMR

Glue

BigQuery

Dataflow

Dataproc

Microsoft SQL

Herramientas

Databricks

Delta Lake

Notebooks

Jobs

Unity Catalog

AWS

GCP

S3

EMR

Glue

BigQuery

Dataflow

Dataproc

GCS

Descripción del empleo

Data Engineer (Spark, Python, AWS/GCP, Databricks)

Experience: 6+ Years

Location: Hyderabad

Required Skills

• Strong experience in Python, PySpark, SQL
• Hands-on experience with AWS + Databricks OR GCP + Databricks
• Experience in building and maintaining ETL/ELT pipelines
• Good understanding of Data Warehousing and Big Data concepts
• Experience with Spark performance tuning and optimization
• Hands-on experience with Delta Lake, Databricks Workflows, Notebooks, Jobs, and Unity Catalog
• Experience with cloud services such as:
• AWS: S3, EMR, Glue, Athena
• GCP: GCS, Dataproc, BigQuery, Dataflow
• Experience working with relational and NoSQL databases

Responsibilities

• Develop scalable data pipelines using Spark and Python.
• Design and implement cloud-based data solutions on AWS or GCP.
• Build, optimize, and maintain ETL processes.
• Collaborate with cross-functional teams to deliver data-driven solutions.
• Ensure data quality, performance, and reliability across platforms.

Keywords: Spark, PySpark, "", SQL, Databricks, AWS, GCP, Delta Lake, ETL, Data Engineering.