Senior Data Engineer (Java | Apache Spark | AWS)

Synthlane; Hyderabad; On-site; INR 3,000,000 - 5,500,000; Full time

Job summary

Synthlane in Hyderabad, India is seeking an experienced Java Spark AWS Data Engineer to design, develop, and maintain scalable data pipelines on AWS. The role requires 8+ years of hands-on experience with Java, Apache Spark, AWS, and big data processing, along with production-grade data solutions.

The candidate will optimize Spark performance, implement ETL/ELT pipelines, and provide production support while collaborating with cross‑functional teams across Data, DevOps, and Architecture.

Qualifications

• 8+ years of total experience in data engineering and related fields.
• Proficient in Java, Spark, and AWS with strong ETL/ELT skills.
• Experience with production-grade data pipelines and Spark optimization.

Responsibilities

• Design and develop scalable data‑processing applications using Java and Apache Spark.
• Build and maintain production‑grade ETL/ELT data pipelines on AWS.
• Develop distributed batch and real‑time data‑processing solutions.
• Process large volumes of structured, semi‑structured, and unstructured data.
• Build Spark applications using Java, Spark SQL, and DataFrame APIs.
• Optimize Spark jobs for performance, memory utilization, partitioning, and scalability.
• Design and manage data ingestion and transformation workflows using AWS services.
• Work with AWS services such as S3, EMR, Glue, Lambda, Athena, Redshift, RDS, and CloudWatch.

Skills

Java

Apache Spark

AWS

SQL

Data pipelines

Data engineering

Tools

Spark SQL

DataFrames

Amazon S3

EMR

Glue

Job description

We are looking for an experienced Java Spark AWS Data Engineer with strong hands‑on expertise in Java, Apache Spark, AWS, SQL, and large‑scale data processing.

The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines and distributed data‑processing applications on AWS. The role requires strong software engineering fundamentals along with practical experience in Spark performance optimization, cloud‑native data services, ETL/ELT pipelines, and production support.
Candidates must have 8+ years of overall experience, with strong hands‑on expertise in Java, Apache Spark, AWS, and large‑scale data engineering solutions.

Key Responsibilities

Role Overview

We are looking for an experienced Java Spark AWS Data Engineer with strong hands‑on expertise in Java, Apache Spark, AWS, SQL, and large‑scale data processing.

The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines and distributed data‑processing applications on AWS. The role requires strong software engineering fundamentals along with practical experience in Spark performance optimization, cloud‑native data services, ETL/ELT pipelines, and production support.
Candidates must have 8+ years of overall experience, with strong hands‑on expertise in Java, Apache Spark, AWS, and large‑scale data engineering solutions.

• Design and develop scalable data‑processing applications using Java and Apache Spark.
• Build and maintain production‑grade ETL/ELT data pipelines on AWS.
• Develop distributed batch and real‑time data‑processing solutions.
• Process large volumes of structured, semi‑structured, and unstructured data.
• Build Spark applications using Java, Spark SQL, and DataFrame APIs.
• Optimize Spark jobs for performance, memory utilization, partitioning, and scalability.
• Design and manage data ingestion and transformation workflows using AWS services.
• Work with AWS services such as Amazon S3, EMR, Glue, Lambda, Athena, Redshift, RDS, and CloudWatch.
• Develop and integrate REST APIs and backend services using Java where required.
• Implement data validation, reconciliation, quality checks, and error‑handling mechanisms.
• Design scalable data models and data warehouse solutions.
• Troubleshoot Spark jobs, data pipeline failures, and production performance issues.
• Implement monitoring, logging, and alerting for data‑processing workloads.
• Participate in system design, architecture discussions, and code reviews.
• Write unit, integration, and data‑pipeline tests.
• Support CI/CD pipelines and automated deployments.
• Collaborate with Data Engineers, Architects, DevOps/SRE teams, and business stakeholders.
• Perform root‑cause analysis and implement permanent fixes for production issues.

Requirements

Java, Java 8, Java 11, Java 17, Apache Spark, Spark SQL, Spark DataFrames, Distributed Data Processing, ETL, ELT, Data Engineering, Data Pipelines, Data Ingestion, Data Transformation, Data Validation, Data Quality, SQL, AWS, Amazon S3, Amazon EMR, AWS Glue, Amazon Athena, Amazon Redshift, AWS Lambda, Amazon RDS, AWS CloudWatch, AWS IAM, Batch Processing, Data Warehousing, Data Modeling, Parquet, JSON, CSV, Spark Optimization, Performance Tuning, Partitioning, Broadcast Joins, Caching, Git, Maven, Gradle, CI/CD, Linux, Production Support, Troubleshooting, Root Cause Analysis