Lead I - Data Engineering

UST; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

UST is seeking a highly skilled Data Engineer to design, develop, and deploy scalable data solutions on AWS. You will build data pipelines with Spark/PySpark, orchestrate ETL workflows, and work across teams to deliver production-ready data products.

The role emphasizes hands-on coding, CI/CD, testing, and optimizing large datasets in a cloud environment, with a focus on reliability and scalable architectures.

Qualifications

• Bachelor's or master's degree in Computer Science, Information Technology, Engineering, or a related field.
• 4+ years of experience in Data Engineering, Big Data, or related roles.
• Strong hands-on experience with Apache Spark and PySpark.
• Experience working with AWS Cloud services for data processing and analytics.
• Solid understanding of Big Data architectures and distributed computing concepts.
• Expertise in designing and implementing ETL pipelines.
• Experience with workflow automation and pipeline orchestration tools.
• Knowledge of CI/CD practices and deployment automation.
• Strong analytical and problem-solving skills with the ability to work with large and complex datasets.
• Experience in functional testing and validation of data applications.
• Strong programming and scripting skills, preferably in Python.

Responsibilities

• Collaborate with cross-functional teams to design, develop, and deploy data tools that support business operations and product initiatives.
• Build and maintain scalable data pipelines and ETL processes using Apache Spark and PySpark.
• Perform analysis and processing of large-scale datasets leveraging AWS cloud services.
• Own product features throughout the software development lifecycle, including design, development, testing, deployment, and production support.
• Evaluate emerging big data technologies and prototype innovative solutions to enhance data processing capabilities and architecture.
• Develop and automate data workflows and pipelines to improve efficiency, reliability, and scalability.
• Implement best practices for CI/CD, code quality, testing, and deployment automation.
• Conduct functional testing and validation of data solutions to ensure accuracy and performance.
• Monitor, troubleshoot, and optimize data processing jobs and workflows.
• Work closely with stakeholders to understand data requirements and translate them into scalable technical solutions.

Skills

Apache Spark

PySpark

AWS

Big Data

ETL development

Workflow automation

CI/CD

Python

Data pipelines

Production deployment

Education

Bachelor's or Master's degree in Computer Science/IT/Engineering

Job description

Who we are:

At UST, we help the world’s best organizations grow and succeed through transformation. Bringing together the right talent, tools, and ideas, we work with our client to co-create lasting change. Together, with over 30,000 employees in 30+ countries, we build for boundless impact—touching billions of lives in the process. Visit us at UST.com.

Role Description

We are seeking a highly skilled Data Engineer to design, develop, and deploy scalable data solutions that support both operational and product-driven business needs. The ideal candidate will have strong expertise in AWS Cloud, Apache Spark/PySpark, ETL development, workflow automation, and big data technologies. This role requires hands‑on experience in building data pipelines, analysing large datasets, optimizing data processing architectures, and driving solutions from development through production deployment.

Key Responsibilities

• Collaborate with cross‑functional teams to design, develop, and deploy data tools that support business operations and product initiatives.
• Build and maintain scalable data pipelines and ETL processes using Apache Spark and PySpark.
• Perform analysis and processing of large‑scale datasets leveraging AWS cloud services.
• Own product features throughout the software development lifecycle, including design, development, testing, deployment, and production support.
• Evaluate emerging big data technologies and prototype innovative solutions to enhance data processing capabilities and architecture.
• Develop and automate data workflows and pipelines to improve efficiency, reliability, and scalability.
• Implement best practices for CI/CD, code quality, testing, and deployment automation.
• Conduct functional testing and validation of data solutions to ensure accuracy and performance.
• Monitor, troubleshoot, and optimize data processing jobs and workflows.
• Work closely with stakeholders to understand data requirements and translate them into scalable technical solutions.

Required Skills

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
• 4+ years of experience in Data Engineering, Big Data, or related roles.
• Strong hands‑on experience with Apache Spark and PySpark.
• Experience working with AWS Cloud services for data processing and analytics.
• Solid understanding of Big Data architectures and distributed computing concepts.
• Expertise in designing and implementing ETL pipelines.
• Experience with workflow automation and pipeline orchestration tools.
• Knowledge of CI/CD practices and deployment automation.
• Strong analytical and problem‑solving skills with the ability to work with large and complex datasets.
• Experience in functional testing and validation of data applications.
• Strong programming and scripting skills, preferably in Python.

Preferred Skills

• Experience with data lake and cloud‑native data architectures.
• Knowledge of containerization technologies and DevOps practices.
• Familiarity with real‑time and batch data processing frameworks.
• Experience in performance tuning and optimization of Spark applications.
• Understanding of data governance, security, and data quality best practices

What We Believe

We’re proud to embrace the same values that have shaped UST since the beginning. Since day one, we’ve been building enduring relationships and a culture of integrity. And today, it's those same values that are inspiring us to encourage innovation from everyone, to champion diversity and inclusion and to place people at the centre of everything we do.

Humility

We will listen, learn, be empathetic and help selflessly in our interactions with everyone.

Humanity

Through business, we will better the lives of those less fortunate than ourselves.

Integrity

We honour our commitments and act with responsibility in all our relationships.

Equal Employment Opportunity Statement

UST is an Equal Opportunity Employer. We believe that no one should be discriminated against because of their differences, such as age, disability, ethnicity, gender, gender identity and expression, religion, or sexual orientation.

All employment decisions shall be made without regard to age, race, creed, colour, religion, sex, national origin, ancestry, disability status, veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by federal, state, or local law.

UST reserves the right to periodically redefine your roles and responsibilities based on the requirements of the organization and/or your performance.

• To support and promote the values of UST.
• Comply with all Company policies and procedures

Skills

Apache Spark, AWS, Big Data, Data Engineering, Workflow Automation, CI/CD, Data Analysis, ETL, Functional Testing, PySpark