Scala and Spark Data Engineer

SWITS DIGITAL Private Limited; Maharashtra; Hybrid; INR 1,200,000 - 1,800,000; Full time

Job summary

SWITS DIGITAL Private Limited in Mumbai is seeking an experienced Scala and Spark Data Engineer with strong functional programming and distributed data processing skills. You will design and build scalable data pipelines using Scala and Apache Spark.

Experience with production-grade data platforms is preferred. The role emphasizes hands-on coding, performance tuning of Spark jobs, and collaboration with cross-functional teams.

Qualifications

• Strong hands-on experience with Scala functional programming.
• Good working knowledge of Apache Spark.

Responsibilities

• Design, develop and maintain scalable data processing applications using Scala and Spark.
• Implement data pipelines and distributed processing solutions.
• Apply functional programming principles to develop clean, reusable and maintainable code.
• Optimize Spark jobs for performance and scalability.
• Work with structured data and SQL-based transformations.
• Troubleshoot performance issues and ensure data processing reliability.
• Collaborate with cross-functional teams to deliver data engineering solutions.
• Participate in technical discussions, code reviews and solution design.

Skills

Scala FP

Apache Spark

Data Engineering

SQL

FP concepts

Higher-order functions

Case classes

Pattern matching

Spark DataFrames

RDDs

Transformations

Performance optimization

Distributed processing

Job description

Job Title: Scala and Spark Data Engineer

Job Location: Mumbai

Work Mode: Hybrid 3 days a week from office

Experience: 5 10 Years

Working Hours: 9:00 AM to 6:00 PM IST

Job Description

We are looking for an experienced Scala and Spark Data Engineer with strong expertise in Scala functional programming and distributed data processing. The ideal candidate should have hands‑on experience developing scalable data engineering solutions using Scala and Apache Spark.

Candidates must demonstrate strong functional programming concepts and practical coding skills. Experience limited to Scala implementation, maintenance, or debugging will not be sufficient.

Key Skills

• Strong Scala functional programming
• Apache Spark
• Data Engineering
• SQL
• Functional programming concepts, immutability and pure functions
• Higher‑order functions and Scala collections
• Option, Some and None
• Case classes and pattern matching
• Spark DataFrames, Datasets and RDDs
• Narrow and wide transformations
• Repartitioning and coalescing
• Distributed data processing and performance optimization

Roles and Responsibilities

• Design, develop and maintain scalable data processing applications using Scala and Spark.
• Implement data pipelines and distributed processing solutions.
• Apply functional programming principles to develop clean, reusable and maintainable code.
• Optimize Spark jobs for performance and scalability.
• Work with structured data and SQL‑based transformations.
• Troubleshoot performance issues and ensure data processing reliability.
• Collaborate with cross‑functional teams to deliver data engineering solutions.
• Participate in technical discussions, code reviews and solution design.

Mandatory Candidate Criteria

• Strong hands‑on experience in Scala functional programming.
• Good working knowledge of Apache Spark.
• Strong understanding of SQL.
• Ability to explain functional programming concepts and demonstrate coding knowledge.
• Candidates whose Scala experience is limited to implementation or debugging are not suitable.

Interview Process

• 1 internal technical round
• 2 client rounds
• An additional client round or face-to-face interview may be required.