Sr. Data Engineer

UST; Hyderabad; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

UST is seeking a Senior Data Engineer to lead data platform initiatives in Hyderabad. You will develop API-driven batch processing systems, build backend data services, and work with AWS, Spark, Python/Scala.

Join a cross-functional Agile team, driving data pipelines, Airflow workflows, and governance across large-scale datasets, ensuring reliability and performance.

Candidates should have 5+ years of hands-on data engineering experience and a strong cloud/data background.

Qualifications

• 5+ years of hands-on experience developing enterprise-scale data platforms or distributed systems.
• Experience with AWS cloud services and Hadoop ecosystem.

Responsibilities

• Develop API-driven systems to govern large-scale batch data applications.
• Build scalable backend data services for data processing workflows.
• Develop and maintain data transformations using Spark, SQL/Hive, Python or Scala.
• Work with AWS services (S3, EC2, EMR, Lambda, DynamoDB) to build cloud-native solutions.
• Design and enhance Airflow workflows with complex DAGs and monitoring.
• Participate in architecture reviews, performance tuning, and DR readiness.
• Collaborate with product owners and cross-functional teams in Agile/Scrum.

Skills

Big Data

Python/Scala

SQL/Hive

Spark

Airflow

AWS

GIT/Jenkins

Distributed systems

Education

Bachelor’s degree in CS/Engineering or related

Tools

AWS EMR

Hadoop ecosystem

Airflow

Git

Jenkins

Job description

Who We Are

At UST, we help the world’s best organizations grow and succeed through transformation. Bringing together the right talent, tools, and ideas, we work with our client to co-create lasting change. Together, with over 30,000 employees in 30+ countries, we build for boundless impact—touching billions of lives in the process. Visit us at .

Role Description

UST Tittle: Lead I - Data Engineering

Who We Are

At UST, we help the world’s best organizations grow and succeed through transformation. Bringing together the right talent, tools, and ideas, we work with our client to co-create lasting change. Together, with over 30,000 employees in 30+ countries, we build for boundless impact—touching billions of lives in the process. Visit us at .

Summary

UST is looking for Sr. Data Engineer

The Opportunity

• Develop API-driven systems to govern, manage, and monitor large-scale batch Big Data applications.
• Build scalable backend services and data engineering solutions that support data processing and operational workflows.
• Develop and maintain data transformation processes using Spark, SQL, Hive, Python, Scala, and related technologies.
• Work with AWS cloud services such as S3, EC2, EMR, Lambda, DynamoDB, and API Gateway to build cloud-native data and backend solutions.
• Build and enhance workflow orchestration using Apache Airflow, including advanced DAG design, dependency management, scheduling, monitoring, and failure handling.
• Define technical scope, objectives, and implementation approaches by participating in requirements gathering, technical research, and process definition.
• Participate in architecture reviews, code reviews, performance tuning, and operational readiness activities.
• Contribute to test planning and validation for application integrations, functional areas, and project deliverables.
• Partner with product owners, data engineers, backend engineers, QA, DevOps, and other cross-functional teams to deliver high-quality solutions in an Agile/Scrum environment.
• Troubleshoot production issues, improve system reliability, and drive automation across data and backend workflows.
• 5+ years of hands-on experience developing enterprise-scale applications, data platforms, or distributed systems.
• 5+ years of experience developing and operating Big Data platforms and cloud-based infrastructure services, preferably using AWS EMR and the Hadoop ecosystem.
• Hands-on experience with AWS, Spark, Python and/or Scala, Airflow, SQL, Hive, and related data processing technologies.
• Proficiency in either Python or Scala, with the ability to build production-grade data pipelines and backend services.
• Experience with AWS cloud services, including S3, EC2, EMR, Lambda, DynamoDB, and API Gateway.
• Strong experience with Apache Airflow orchestration, including complex workflows, scheduling, dependency handling, monitoring, and operational support.
• Strong understanding of data engineering concepts, including data pipelines, data transformation, batch processing, real‑time processing, data quality, and performance optimization.
• Excellent analytical and problem‑solving skills.
• Strong communication and presentation skills, with the ability to explain technical concepts clearly to both technical and non‑technical audiences.
• Experience with version control and CI/CD tools, including Git and Jenkins, for source code management, build automation, testing, and deployment workflows. Ability to work effectively in cross‑functional Agile/Scrum teams.
• Bachelor’s degree in computer science, Engineering, Mathematics, Information Systems, or a related technical field, or equivalent practical experience.

Preferred Qualifications

• Experience with LLMs, Generative AI, Agentic AI, or AI‑assisted engineering workflows.
• Experience with API design, microservices, event‑driven architecture, or serverless backend development.
• Experience with CI/CD pipelines, infrastructure as code, automated testing, and production deployment practices.
• Experience with data governance, metadata management, lineage, data observability, or operational controls for enterprise data platforms

What We Believe

We’re proud to embrace the same values that have shaped UST since the beginning. Since day one, we’ve been building enduring relationships and a culture of integrity. And today, it's those same values that are inspiring us to encourage innovation from everyone, to champion diversity and inclusion and to place people at the centre of everything we do.

Humility

We will listen, learn, be empathetic and help selflessly in our interactions with everyone.

Humanity

Through business, we will better the lives of those less fortunate than ourselves.

Integrity

We honour our commitments and act with responsibility in all our relationships.

Equal Employment Opportunity Statement

UST is an Equal Opportunity Employer. We believe that no one should be discriminated against because of their differences, such as age, disability, ethnicity, gender, gender identity and expression, religion, or sexual orientation.

All employment decisions shall be made without regard to age, race, creed, colour, religion, sex, national origin, ancestry, disability status, veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by federal, state, or local law.

UST reserves the right to periodically redefine your roles and responsibilities based on the requirements of the organization and/or your performance.

• To support and promote the values of UST.
• Comply with all Company policies and procedures

Skills

AWS, AWS Lambda