Big Data Engineer Lead

Evoke HR; Chennai District, Bengaluru; Hybrid; INR 2,500,000 - 4,000,000; Full time

Job summary

Evoke HR seeks an experienced Big Data Engineer Lead in India to drive scalable data solutions. You will work hands-on with PySpark, Spark, Python, SQL, Hive and Unix, and provide technical leadership to the team.

Design and optimize data pipelines, mentor engineers, review code, and collaborate with stakeholders to translate requirements into robust data architectures. Prior lead/senior engineer experience required.

Qualifications

• Experience with large-scale data processing and distributed systems.
• Strong leadership and mentorship capabilities.
• Excellent communication and collaboration skills.

Responsibilities

• Design, build, and maintain Big Data pipelines using Spark and PySpark.
• Develop data processing solutions with Python and SQL.
• Work with Hive for data storage and querying.
• Operate in Unix/Linux environments and troubleshoot issues.
• Lead technical discussions, mentor team members, and review code.
• Analyze requirements and deliver scalable data solutions.
• Address performance and production issues promptly.

Skills

PySpark

Apache Spark

Python

SQL

Hive

Unix/Linux

ETL/ELT

Big Data processing

Lead / Senior Engineer

Team leadership

Job description

We are looking for an experienced Big Data Engineer Lead with strong hands-on expertise in PySpark, Python, SQL, Spark, Hive, and Unix. The candidate will be responsible for developing scalable data solutions and providing technical leadership to the team.

Key Responsibilities

• Design, develop, and maintain Big Data pipelines using Spark and PySpark.
• Develop data processing and transformation solutions using Python and SQL.
• Work with Hive for data storage, querying, and processing.
• Perform data processing and troubleshooting in Unix/Linux environments.
• Lead technical discussions, mentor team members, and conduct code reviews.
• Analyze requirements and deliver scalable and optimized data solutions.
• Troubleshoot performance and production issues and ensure timely resolution.

Required Skills

• Strong hands-on experience in PySpark and Apache Spark.
• Excellent knowledge of Python and SQL.
• Experience with Hive and Unix/Linux.
• Strong understanding of ETL/ELT and Big Data processing.
• Experience in a Lead / Senior Engineer role with good team-handling skills.
• Strong analytical, problem-solving, and communication skills.