Associate III - Data Engineering

UST; Hyderabad; On-site; INR 1,200,000 - 2,300,000; Full time

Job summary

UST is seeking a Big Data Engineer with 2-4 years of experience to design, develop, and maintain large-scale data processing systems on AWS. The role focuses on PySpark, Python, and building scalable pipelines and data lakes to support analytics and BI initiatives.

You will collaborate with data architects, analysts, and business stakeholders, implementing secure, high-quality data solutions and ensuring data availability. Prior experience with AWS services and data warehousing is essential.

Qualifications

• 2-4 years of experience in designing, developing, and maintaining data processing systems.
• Strong expertise in AWS, Python, and PySpark for scalable pipelines and data lakes.
• Experience building ETL/ELT workflows and working with data warehouses on AWS.

Responsibilities

• Design, develop, and maintain scalable data pipelines for large data volumes.
• Build and optimize ETL/ELT workflows using PySpark and Python.
• Develop and manage data lake and data warehouse solutions on AWS.

Skills

Python

PySpark

AWS

Data pipelines

SQL

Data warehousing

Git

Tools

Amazon S3

AWS Glue

Amazon EMR

AWS Lambda

Amazon Redshift

Amazon Athena

AWS IAM

CloudWatch

Job description

Who we are:

At UST, we help the world’s best organizations grow and succeed through transformation. Bringing together the right talent, tools, and ideas, we work with our client to co-create lasting change. Together, with over 30,000 employees in 30+ countries, we build for boundless impact—touching billions of lives in the process. Visit us at .

Summary

We are seeking a skilled and motivated Big Data Engineer with 2-4 years of experience in designing, developing, and maintaining large-scale data processing systems. The ideal candidate should have strong expertise in AWS cloud services, Python, and PySpark, along with hands‑on experience in building scalable data pipelines, data lakes, and ETL solutions. The candidate will work closely with data architects, analysts, and business stakeholders to deliver high-quality data solutions that support analytics and business intelligence initiatives.

Key Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines for processing large volumes of structured and unstructured data.
• Build and optimize ETL/ELT workflows using PySpark and Python.
• Develop and manage data lake and data warehouse solutions on AWS Cloud.
• Implement data ingestion frameworks from multiple data sources, including databases, APIs, files, and streaming platforms.
• Leverage AWS services such as S3, Glue, EMR, Lambda, Redshift, Athena, CloudWatch, and IAM for data engineering solutions.
• Perform data transformation, cleansing, validation, and enrichment to ensure high data quality.
• Optimize Spark applications for performance, scalability, and cost efficiency.
• Collaborate with data scientists, analysts, and business teams to understand data requirements and deliver solutions.
• Monitor data pipelines and resolve production issues to ensure data availability and reliability.
• Implement security best practices, governance, and compliance standards across data platforms.
• Participate in code reviews, unit testing, and deployment activities.
• Create technical documentation and maintain operational runbooks.

Required Skills

Technical Skills

• Strong experience in Python programming.
• Hands-on expertise in PySpark and distributed data processing.
• Experience working with AWS Cloud Platform.
• Knowledge of AWS services including:
  • Amazon S3
  • AWS Glue
  • Amazon EMR
  • AWS Lambda
  • Amazon Redshift
  • Amazon Athena
  • AWS IAM
  • CloudWatch
• Good understanding of ETL/ELT concepts and data pipeline development.
• Experience with SQL and relational databases.
• Knowledge of Data Warehousing concepts and dimensional modeling.
• Familiarity with version control tools such as Git.

AWS, Data Engineering, PySpark