Lead Data Engineer (Spark/Scala/Python)

TechDigital Group; Houston (TX); On-site; USD 100,000 - 140,000; Full time

Job summary

A leading technology firm in the United States is seeking a Senior Technical Lead to oversee technical teams in the development of scalable solutions using Apache Spark, Scala, and Python. The ideal candidate will be proficient in these technologies and have experience in leading projects while guiding team members through complex issues. Strong communication and leadership skills are essential in this collaborative role.

Qualifications

• Strong proficiency in Apache Spark, Scala, and Python.
• Experience in leading technical teams and projects.
• Excellent problem-solving skills with critical thinking.

Responsibilities

• Lead technical teams in the design and implementation of solutions.
• Provide technical expertise and guidance to team members.
• Collaborate with stakeholders to gather requirements.
• Conduct code reviews and optimize performance.

Skills

Apache Spark

Scala

Python

Problem-solving

Software development life cycle

Leadership

Performance tuning

Performance tuning

Education

Relevant certifications in Apache Spark, Scala, or Python

Job description

Overview

Senior Technical Lead with Apache Spark, Scala, Python. The Senior Technical Lead will be responsible for leading technical teams and projects related to Apache Spark, Scala, and Python. The role involves overseeing the design, development, and implementation of scalable and efficient solutions using these technologies.

Key Responsibilities

• Lead technical teams in the design and implementation of solutions using Apache Spark, Scala, and Python
• Provide technical expertise and guidance to team members in resolving complex technical issues
• Collaborate with stakeholders to gather requirements and define project scope
• Ensure adherence to best practices in coding, testing, and deployment processes
• Conduct code reviews and performance optimization activities
• Troubleshoot and debug technical issues to ensure seamless project delivery
• Stay updated with the latest trends and advancements in Apache Spark, Scala, and Python technologies
• Mentor team members and facilitate knowledge sharing within the team

Skill Requirements

• Strong proficiency in Apache Spark, Scala, and Python
• Experience in leading technical teams and projects
• Excellent problem-solving skills and ability to think critically
• Solid understanding of the software development life cycle
• Good communication and interpersonal skills
• Ability to work effectively in a collaborative team environment
• Strong organizational and leadership abilities
• Experience in performance tuning and optimization techniques
• Knowledge of big data concepts and tools is a plus

Certifications

Relevant certifications in Apache Spark, Scala, or Python are a plus