Lead Data Engineer

Placements24; Sandton; Hybrid; ZAR 1,200,000 - 1,800,000; Full time

Benefits offered by this job

Performance bonuses

Health insurance

Retirement plan

Paid time off

Remote work with flexible hours

Professional development budget

Conferences access

Access to advanced data tooling

Job summary

Placements24 is seeking a Lead Data Engineer to architect and build robust data pipelines and infrastructure in a fully remote role. You will lead a team, design scalable data solutions, and ensure data flows support analysts and scientists across the organization.

You will leverage Spark, Kafka, and cloud data platforms to deliver high-availability data services, while mentoring junior engineers and driving best-practices in governance and security. Remote-first culture emphasized.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or related field.
• Minimum 7 years in data engineering with lead/architect experience.
• Proven design of large-scale data pipelines using Spark, Kafka, Flink, etc.
• Proficient in Python, Scala, or Java; strong SQL and various DB technologies.
• Experience with cloud data platforms (AWS/Azure/GCP) and data warehousing concepts.
• Familiar with Docker/Kubernetes; CI/CD is a plus.
• Excellent leadership and communication skills; remote-work capable.

Responsibilities

• Lead design, development, and maintenance of scalable data pipelines and ETL/ELT processes.
• Architect data warehousing and data lake strategies.
• Develop data infrastructure with high availability, performance, and security.
• Collaborate with data scientists, analysts and stakeholders to meet data needs.
• Define and enforce data governance policies and best practices.
• Optimize data processing workflows and query performance.
• Select and implement data technologies and tools.
• Mentor junior data engineers and promote technical excellence.
• Monitor systems, identify issues, and implement solutions.
• Stay current with emerging data engineering trends and tools.

Skills

Leadership

Data governance

Python/Scala/Java

SQL

Cloud platforms

Spark/Kafka/Flink

Data warehousing

Docker/Kubernetes

CI/CD

Remote collaboration

Education

Bachelor's or Master's in CS/Engineering

Tools

S3/Redshift/BigQuery/Snowflake

Job description

About the Role

Our client is seeking an accomplished and forward-thinking Lead Data Engineer to architect and build robust data pipelines and infrastructure. This is a fully remote position, ideal for an experienced leader who can guide a team, design scalable data solutions, and ensure the efficient flow of data across the organization. You will leverage cutting-edge technologies to build and maintain our data platform, enabling data scientists and analysts to derive critical insights. Working remotely from Sandton, Gauteng, ZA , you will collaborate with distributed teams, driving best practices in data management, governance, and quality. We foster a remote-first culture that values autonomy, innovation, and continuous professional growth. This is a pivotal role in shaping our data capabilities and empowering data-driven decision-making at all levels.

Key Responsibilities

• Lead the design, development, and maintenance of scalable and reliable data pipelines and ETL/ELT processes.


• Architect and implement data warehousing solutions and data lake strategies.


• Develop and manage data infrastructure, ensuring high availability, performance, and security.


• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions.


• Define and enforce data governance policies, standards, and best practices.


• Optimize data processing workflows and query performance.


• Select and implement appropriate data technologies and tools.


• Mentor and guide junior data engineers, fostering a culture of technical excellence.


• Monitor data systems, identify issues, and implement effective solutions.


• Stay current with emerging trends and technologies in data engineering and big data.

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.


• Minimum of 7 years of professional experience in data engineering, with a significant portion in a lead or architect role.


• Proven experience in designing and building large-scale data pipelines using technologies like Spark, Kafka, Flink, etc.


• Proficiency in programming languages such as Python, Scala, or Java.


• Strong SQL expertise and experience with various database technologies (relational, NoSQL, data warehouses).


• Experience with cloud data platforms (AWS, Azure, GCP), including services like S3, Redshift, BigQuery, Snowflake.


• Familiarity with data modeling, ETL/ELT processes, and data warehousing concepts.


• Experience with containerization (Docker, Kubernetes) and CI/CD is a plus.


• Excellent leadership, communication, and interpersonal skills.


• Ability to work independently and manage projects effectively in a remote environment.

Benefits

• Highly competitive salary and performance bonuses.


• Comprehensive health, dental, and vision insurance.


• Retirement savings plan with company match.


• Generous paid time off and flexible leave policies.


• Fully remote work arrangement with flexible hours.


• Budget for professional development, certifications, and conferences.


• Access to advanced data engineering tools and technologies.


• A collaborative, innovative, and inclusive team environment.


• Opportunity to shape the future of our data architecture.