Lead I - Data Engineering - Java Spark

UST; Thiruvananthapuram; On-site; INR 2,000,000 - 3,600,000; Full time

Job summary

UST is seeking a Data Engineer proficient in Java and Spark to design, develop, and maintain scalable enterprise data solutions. You will build high-performance batch and streaming pipelines, implement Kafka-based streaming architectures, and work with Hadoop, Hive, and cloud storage such as Amazon S3.

Ideal candidates have core Java mastery, Spark SQL, DataFrames, Datasets, and Spark Streaming, plus strong SQL skills and experience with distributed data environments.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related technical field.
• 5+ years of professional software engineering experience.
• Strong hands-on experience in enterprise Core Java application development.
• Proven experience with Apache Spark and Big Data technologies.
• Experience implementing real-time messaging and streaming solutions.

Responsibilities

• Design and develop enterprise-grade applications using Core Java.
• Build and maintain scalable data pipelines using Apache Spark.
• Develop batch and real-time data processing solutions using Spark SQL, DataFrames, Datasets, and Spark Streaming.
• Implement event-driven and streaming architectures using Apache Kafka.
• Write, optimize, and tune complex SQL queries for high-performance applications.
• Work with distributed storage systems and Big Data platforms such as Hadoop HDFS, Hive, and Amazon S3.
• Develop integrations with NoSQL databases including HBase and Cassandra.
• Collaborate with cross-functional teams to design scalable and reliable data solutions.
• Troubleshoot, optimize, and enhance application performance and reliability.

Skills

Core Java

Spark SQL

DataFrames

Datasets

Spark Streaming

Apache Kafka

Advanced SQL

Hadoop HDFS

Hive

Amazon S3

HBase or Cassandra

Distributed Systems

Education

Bachelor's or Master's in CS/IT/Engineering

Tools

Hadoop

Job description

Role Description

Who we are:
At UST, we help the world’s best organizations grow and succeed through transformation. Bringing together the right talent, tools, and ideas, we work with our clients to co-create lasting change. Together, with over 30,000 employees in 30+ countries, we build for boundless impact, touching billions of lives in the process. Visit us at UST.com.

Summary

UST is looking for a highly skilled Data Engineer - Java + Spark with strong experience in enterprise application development, Big Data technologies, and real-time data processing solutions. The ideal candidate should have expertise in Core Java, Apache Spark, Kafka, and SQL, along with experience working in distributed data environments.

The Opportunity

Job Description

We are seeking an experienced Java Spark Developer to design, develop, and maintain scalable enterprise applications and data processing solutions. The candidate will be responsible for building high-performance Big Data applications, implementing real-time streaming architectures, and optimizing large-scale data processing systems.

Key Roles & Responsibilities

• Design and develop enterprise-grade applications using Core Java.
• Build and maintain scalable data pipelines using Apache Spark.
• Develop batch and real-time data processing solutions using Spark SQL, DataFrames, Datasets, and Spark Streaming.
• Implement event-driven and streaming architectures using Apache Kafka.
• Write, optimize, and tune complex SQL queries for high-performance applications.
• Work with distributed storage systems and Big Data platforms such as Hadoop HDFS, Hive, and Amazon S3.
• Develop integrations with NoSQL databases including HBase and Cassandra.
• Collaborate with cross-functional teams to design scalable and reliable data solutions.
• Troubleshoot, optimize, and enhance application performance and reliability.
• Follow software engineering best practices and contribute to architecture discussions.

What You Need

Required Skills

• Core Java
• Apache Spark
  • Spark SQL
  • DataFrames
  • Datasets
  • Spark Streaming
• Apache Kafka
• Advanced SQL Development and Query Optimization
• Hadoop HDFS
• Hive
• Amazon S3
• HBase or Cassandra
• Distributed Systems and Big Data Architecture

Desired Skills

• RabbitMQ
• AWS Kinesis
• Experience with large-scale distributed data processing platforms
• Performance tuning and optimization of Big Data applications
• Knowledge of cloud-based data architectures

Qualification

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related technical field.
• 5+ years of professional software engineering experience.
• Strong hands‑on experience in enterprise Core Java application development.
• Proven experience with Apache Spark and Big Data technologies.
• Experience implementing real-time messaging and streaming solutions.

What We Believe

We're proud to embrace the same values that have shaped UST since the beginning. Since day one, we've been building enduring relationships and a culture of integrity. Today, those same values inspire us to encourage innovation from everyone, champion diversity and inclusion, and place people at the center of everything we do.

Humility

We will listen, learn, be empathetic, and help selflessly in our interactions with everyone.

Humanity

Through business, we will better the lives of those less fortunate than ourselves.

Integrity

We honour our commitments and act with responsibility in all our relationships.

Equal Employment Opportunity Statement

UST is an Equal Opportunity Employer. We believe that no one should be discriminated against because of their differences, such as age, disability, ethnicity, gender, gender identity and expression, religion, or sexual orientation.

All employment decisions shall be made without regard to age, race, creed, colour, religion, sex, national origin, ancestry, disability status, veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status, or any other basis as protected by federal, state, or local law.

UST reserves the right to periodically redefine your roles and responsibilities based on the requirements of the organization and/or your performance.

• To support and promote the values of UST.
• Comply with all Company policies and procedures.

Skills

Apache Kafka, Apache Spark, Java, SQL