Data platform Engineer

UST

Hyderabad

On-site

INR 1,200,000 - 1,800,000

Full time

6 days ago

Be an early applicant

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

UST is seeking an experienced Data Engineer to design and maintain scalable data pipelines and cloud infrastructure in Hyderabad. The role emphasizes Python and PySpark for large-scale data processing and hands-on AWS expertise across S3, EMR, Glue, and Redshift.

You will also implement Terraform-based IaC, CI/CD for data workflows, and collaborate with data scientists, analysts, and stakeholders to ensure data quality, security, and governance. A strong coding and cloud background is essential.

Qualifications

6+ years of experience in data engineering or a related field.
Strong proficiency in Python for data processing and automation.
Hands-on experience with PySpark for large-scale distributed data processing.
Solid experience with Terraform for infrastructure as code.
Strong expertise in AWS services (S3, EMR, Glue, Redshift, Lambda, IAM).
Experience with SQL and working with relational/non-relational databases.

Responsibilities

Design, build, and maintain robust, scalable data pipelines for batch and streaming data processing.
Develop and optimize ETL/ELT workflows using Python and PySpark.
Architect and manage cloud infrastructure on AWS (S3, EMR, Glue, Lambda, Redshift, EC2, IAM).
Use Terraform to provision, manage, and version-control infrastructure as code (IaC).
Collaborate with data scientists, analysts, and business stakeholders to deliver reliable data solutions.

Skills

Python

PySpark

Terraform

AWS

SQL

Airflow

CI/CD

Data Modeling

Git

Kafka

Docker

Kubernetes

Tools

Airflow

Docker

Kubernetes

Git

CI/CD Tools

Job description

At UST, we help the world’s best organizations grow and succeed through transformation. Bringing together the right talent, tools, and ideas, we work with our clients to co-create lasting change. Together, with over 30,000 employees in 30+ countries, we build for boundless impact, touching billions of lives in the process.

Role Description

Summary

We are looking for an experienced Data Engineer with a strong background in building and maintaining scalable data pipelines and cloud infrastructure. The ideal candidate will have hands‑on expertise in Python, PySpark, Terraform, and AWS, along with a solid understanding of data engineering best practices.

Senior Data Platform Engineer

Experience: 6+ Years Location: Hyderabad

Key Responsibilities

Design, build, and maintain robust, scalable, and efficient data pipelines for batch and streaming data processing. Develop and optimize ETL/ELT workflows using Python and PySpark to process large-scale datasets.
Architect and manage cloud infrastructure on AWS (S3, EMR, Glue, Lambda, Redshift, EC2, IAM, etc.).
Use Terraform to provision, manage, and version‑control infrastructure as code (IaC). Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver reliable data solutions.
Ensure data quality, integrity, and security across pipelines and storage systems.
Monitor and troubleshoot production data pipelines, ensuring high availability and performance.
Implement CI/CD pipelines for data engineering workflows.
Document technical designs, data flows, and architecture for internal reference.
Mentor junior data engineers and contribute to best practices within the team.

Required Skills & Qualifications

6+ years of experience in data engineering or a related field. Strong proficiency in Python for data processing and automation.
Hands‑on experience with PySpark for large-scale distributed data processing.
Solid experience with Terraform for infrastructure as code.
Strong expertise in AWS services (S3, EMR, Glue, Redshift, Lambda, IAM, CloudWatch, etc.).
Experience with SQL and working with relational/non‑relational databases.
Familiarity with orchestration tools like Airflow or similar.
Understanding of data warehousing concepts and data modeling. Experience with version control (Git) and CI/CD pipelines.
Strong problem‑solving skills and ability to work independently.
Good to Have Experience with Kafka or other streaming platforms.
Knowledge of containerization tools (Docker, Kubernetes).
Exposure to data governance and security best practices. AWS certification(s).

What We Believe

We’re proud to embrace the same values that have shaped UST since the beginning. Since day one, we’ve been building enduring relationships and a culture of integrity. Today, those same values inspire us to encourage innovation from everyone, champion diversity and inclusion, and place people at the center of everything we do.

Humility

We will listen, learn, be empathetic, and help selflessly in our interactions with everyone.

Humanity

Through business, we will better the lives of those less fortunate than ourselves.

Integrity

We honour our commitments and act with responsibility in all our relationships.

Equal Employment Opportunity Statement

UST is an Equal Opportunity Employer. We believe that no one should be discriminated against because of their differences, such as age, disability, ethnicity, gender, gender identity and expression, religion, or sexual orientation.

All employment decisions shall be made without regard to age, race, creed, colour, religion, sex, national origin, ancestry, disability status, veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status, or any other basis protected by federal, state, or local law.

UST reserves the right to periodically redefine your roles and responsibilities based on the requirements of the organization and/or your performance.