Senior Apache Spark Technical Lead - Scala, Python

HCLTech; Bengaluru; On-site; INR 3,000,000 - 5,000,000; Full time

Job summary

HCL Technologies Limited is seeking a Senior Apache Spark Technical Lead to guide complex data solutions using Scala and Python. The role emphasizes hands-on development, architectural guidance, and effective team mentorship in Bengaluru.

You will lead technical teams, collaborate with stakeholders, ensure best practices across coding, testing, and deployment, and stay ahead of evolving Spark/Scala/Python ecosystems to deliver scalable data platforms.

Qualifications

• Must have hands-on experience with Spark/Scala/Python in large-scale data environments.
• Proven ability to lead technical teams and deliver complex data solutions.
• Strong understanding of software development lifecycle and code reviews.

Responsibilities

• Lead technical design and implementation using Spark, Scala, and Python.
• Provide guidance to team members to resolve complex issues.
• Collaborate with stakeholders to gather requirements and define project scope.
• Ensure adherence to coding, testing, and deployment best practices.
• Conduct code reviews and optimize performance.
• Troubleshoot issues to ensure timely delivery.
• Stay updated with Spark/Scala/Python trends and advancements.
• Mentor team members and share knowledge across the team.

Skills

Spark

Scala

Python

Tech leadership

Problem solving

SDLC

Communication

Teamwork

Perf tuning

Job description

Career Opportunities: Senior Apache Spark Technical Lead - Scala, Python (132236)

Requisition ID132236-Posted11/10/2026

Hands on development work on Scala/Spark, Hadoop, platform operations with Quantexa Certified Experience.

• Data integration – ETL tools like IBM Datastage and Informatica. Ingestion mechanism like Flume & Kafka.
• Data modelling – Dimensional modelling using RDBMS, NO-SQL and Big Data technologies.
• Big data – Hadoop eco-system, Distributions like Cloudera / Hortonworks, Pig and HIVE.
• Data processing frameworks – Scala/python/Spark.
• Hands-on experience with databases like Oracle, MS SQL Server, NOSQL (HBase / Cassandra, MongoDB)
• Data Platform administration for Hadoop, Elastic, Oracle.
• Exposure to containers, kubernetes, application setup (load balancer), logging, Prometheus

Key Responsibilities

• 1. Lead technical teams in the design and implementation of solutions using apache spark, scala, and python
• 2. Provide technical expertise and guidance to team members in resolving complex technical issues
• 3. Collaborate with stakeholders to gather requirements and define project scope
• 4. Ensure adherence to best practices in coding, testing, and deployment processes
• 5. Conduct code reviews and performance optimization activities
• 6. Troubleshoot and debug technical issues to ensure seamless project delivery
• 7. Stay updated with the latest trends and advancements in apache spark, scala, and python technologies
• 8. Mentor team members and facilitate knowledge sharing within the team

Skill Requirements

• 1. Strong proficiency in apache spark, scala, and python
• 2. Experience in leading technical teams and projects
• 3. Excellent problem-solving skills and ability to think critically
• 4. Solid understanding of software development life cycle
• 5. Good communication and interpersonal skills
• 6. Ability to work effectively in a collaborative team environment
• 7. Strong organizational and leadership abilities
• 8. Experience in performance tuning and optimization techniques
• 9. Knowledge of big data concepts and tools is a plus

Other Requirements

• 1.Relevant certifications in apache spark, scala, or python are a plus