Data Software Engineer – Spark, Python, Databricks (L2 / L4)

Talpro; Bengaluru; On-site; INR 2,100,000 - 4,200,000; Full time

Job summary

A leading technology recruitment firm in India is seeking Data Software Engineers with expertise in Spark, Python, and Databricks. This role requires strong experience in Big Data engineering and the ability to manage complex data platforms. Candidates should have a proven track record with streaming systems, ETL processes, and be adept in Agile environments. A hybrid work model is offered along with competitive salaries based on experience levels.

Qualifications

• Strong hands-on experience in Python for data transformations.
• Deep Big Data engineering experience in distributed systems.
• 4-5 years of experience for L2 and 8-12 years for L4.

Responsibilities

• Design and build distributed data processing systems using Spark.
• Develop and optimize Spark applications for performance.
• Create ETL/ELT pipelines for large-scale data ingestion.

Skills

Python

Distributed computing fundamentals

Spark Streaming

Kafka

Advanced SQL

Performance tuning of Spark jobs

Experience working in Agile

Job description

Data Software Engineer – Spark, Python, Databricks (L2 / L4)

Talpro is leading the way in transforming the talent acquisition landscape. We deliver innovative, sustainable, and cost‑effective recruitment solutions tailored to today’s business needs. Our mission is to offer comprehensive hiring strategies that address immediate recruitment demands while laying the groundwork for long‑term success.

Job Description

Experience:

• L2: 4–5 years
• L4: 8–12 years

Mode: FTE (Full-Time Employment)

Work Mode: Hybrid

Notice Period:

• L2: Immediate to 15 days
• L4: Immediate to 15 days

Drive Type: F2F

CTC Band:

• L2: Up to 21 LPA
• L4: Up to 42 LPA

Role Overview

We are hiring Data Software Engineers with strong expertise in Apache Spark, Python, and AWS/Azure Databricks. The ideal candidates will have deep Big Data engineering experience, strong distributed systems knowledge, and the ability to work on complex end‑to‑end data platforms at scale.

Key Responsibilities

• Design and build distributed data processing systems using Spark and Hadoop.
• Develop and optimize Spark applications, ensuring performance and scalability.
• Create and manage ETL/ELT pipelines for large‑scale data ingestion and transformation.

Streaming & Event Processing

• Build and manage real‑time streaming systems using Spark Streaming or Storm.
• Work with Kafka / RabbitMQ for event‑driven ingestion and messaging patterns.
• Develop & optimize workloads on AWS Databricks or Azure Databricks.
• Perform cluster management, job scheduling, performance tuning, and automation.

Data Integration & Storage

• Integrate data from diverse sources: RDBMS (Oracle, SQL Server), ERP, file systems.
• Work with query engines like Hive and Impala.
• Experience with NoSQL stores: HBase, Cassandra, MongoDB.

Programming & Scripting

• Strong hands‑on coding in Python for data transformations and automations.
• Strong SQL skills for data validation, tuning, and complex queries.
• Provide technical leadership and mentoring to junior engineers.

Ways of Working

• Work in Agile teams, participate in sprint ceremonies and planning.
• Collaborate with engineering, data science, and product teams.

Required Skills & Expertise (Both L2 & L4)

• Python – Strong hands‑on
• Distributed computing fundamentals
• Streaming systems: Spark Streaming / Storm
• Messaging: Kafka or RabbitMQ
• SQL – Advanced (joins, stored procedures, query optimization)
• Performance tuning of Spark jobs
• Experience working in Agile

Experience & Level Mapping

L2 – Mid-Level (4–5 Yrs)

• Skills: Spark, Python, AWS
• Notice Period: Immediate – 20 Days
• CTC Band: Up to 21 LPA

L4 – Senior-Level (8–12 Yrs)