AWS Databricks Spark Scala Developer

Tata Consultancy Services; Chennai District, Bengaluru; On-site; INR 3,000,000 - 3,800,000; Full time

Job summary

Tata Consultancy Services in Bengaluru is seeking an AWS Databricks Spark Scala Developer with 6 to 8 years of data engineering experience to design and implement large-scale processing solutions using Spark, Scala, Databricks, and AWS.

The role emphasizes building ETL/ELT pipelines, deploying JAR-based apps, tuning performance, and collaborating with analysts and architects to deliver scalable cloud data engineering outcomes.

Qualifications

• 6+ years experience with data engineering and Spark-based processing
• Proven ability to design and implement scalable data pipelines
• Hands-on experience building and deploying JAR-based applications in cloud environments
• Strong SQL skills and understanding of data warehousing concepts
• Experience with Git and CI/CD tooling
• Familiarity with Databricks on AWS and Delta Lake (preferred)
• Knowledge of distributed computing and performance tuning

Responsibilities

• Design, develop, and maintain scalable data processing applications using Scala and Spark
• Develop and deploy JAR-based applications in Databricks and AWS environments
• Build ETL/ELT pipelines for large data volumes
• Optimize Spark jobs for performance and reliability
• Implement data engineering solutions using Databricks on AWS
• Collaborate with analysts, architects, and stakeholders to gather requirements
• Monitor, troubleshoot, and resolve production data pipeline issues
• Participate in code reviews and enforce coding standards

Skills

Scala Programming

Apache Spark

Databricks Platform

AWS Cloud Services

JAR Development

SQL

ETL/Data Warehousing

Git/CI-CD

Debugging/Problem Solving

Tools

Databricks Platform

Apache Spark

AWS (S3, EC2, Lambda, IAM, Glue)

Job description

Key Responsibilities

Job Title: AWS Databricks Spark Scala Developer

Experience: 7 to 10 Years
Location: Bangalore / Hyderabad

Job Description

We are seeking a highly skilled AWS Databricks Spark Scala Developer with 6 to 8 years of experience in designing, developing, and implementing large-scale data processing solutions. The ideal candidate should have strong expertise in AWS, Databricks, Apache Spark, Scala, and JAR application development with hands‑on experience in building scalable data engineering solutions.

Key Responsibilities

• Design, develop, and maintain scalable data processing applications using Scala and Apache Spark.
• Develop and deploy JAR-based applications in Databricks and AWS environments.
• Build ETL/ELT pipelines for processing large volumes of structured and unstructured data.
• Optimize Spark jobs for performance, scalability, and reliability.
• Implement data engineering solutions using Databricks on AWS.
• Collaborate with business analysts, architects, and stakeholders to understand requirements and deliver solutions.
• Monitor, troubleshoot, and resolve production issues related to data pipelines.
• Participate in code reviews and ensure adherence to coding standards and best practices.

Required Skills

• Strong experience in Scala Programming
• Hands‑on expertise in Apache Spark (Core, SQL, DataFrames)
• Experience with Databricks Platform
• Strong knowledge of AWS Cloud Services (S3, EC2, Lambda, IAM, Glue)
• Experience in JAR Development and Deployment
• Strong SQL skills and database concepts
• Experience with ETL and Data Warehousing concepts
• Hands‑on experience with Git and CI/CD tools
• Knowledge of distributed computing and performance tuning
• Strong debugging and problem‑solving skills

Preferred Skills

• Experience with Delta Lake
• Knowledge of Apache Kafka
• Exposure to PySpark
• Familiarity with Airflow or AWS Step Functions
• Experience with DevOps and Infrastructure as Code
• Understanding of Data Governance and Data Security practices