Principal Data Engineer

Placements24; Sandton; Hybrid; ZAR 1,200,000 - 1,800,000; Full time

Benefits offered by this job

Bonus structure

Health benefits

Professional development

Fully remote work

Stock options/Equity

Home office budget

Job summary

Placements24 is seeking a Principal Data Engineer for a remote-first role to design, build and maintain scalable data pipelines and infrastructure powering advanced analytics and AI initiatives. This senior position emphasizes leadership and hands-on technical excellence in a distributed team.

You will mentor engineers, set technical direction, collaborate with data scientists and software engineers, and champion cloud-native architectures with AWS, Azure or GCP.

Qualifications

• Master's or PhD in CS/Engineering or related field, or equivalent experience.
• 8+ years in data engineering or related field.
• Proven experience with large-scale data pipelines (Spark, Flink, Kafka).
• Cloud platforms AWS/Azure/GCP and their data services (S3, Redshift, BigQuery, Snowflake, Databricks).
• Strong programming in Python, Scala or Java.
• Extensive SQL and NoSQL experience; data modeling; ETL/ELT.
• Experience with Docker/Kubernetes and CI/CD practices.
• Strong leadership and remote collaboration skills.

Responsibilities

• Design, develop and optimize large-scale batch and real-time data pipelines.
• Build and maintain scalable data infrastructure using cloud-native technologies.
• Collaborate with data scientists, analysts and software engineers to meet data needs.
• Implement data governance, quality, security and performance best practices.
• Develop and manage data warehouses and data lakes.
• Evaluate and select appropriate data technologies and tools.
• Automate data processes and workflows; troubleshoot complex issues.
• Mentor junior data engineers and drive technical growth.
• Stay current with data engineering trends and big data technologies.

Skills

Spark

Kafka

Python

Scala/Java

SQL/NoSQL

Cloud platforms

Docker/Kubernetes

CI/CD

Education

Master's or PhD or equivalent

Tools

Databricks

Snowflake

BigQuery

Job description

About the Role

Our client is a trailblazing data analytics and AI company that is looking for a highly experienced Principal Data Engineer to join their fully remote team. This senior role is instrumental in designing, building, and maintaining the scalable and robust data pipelines and infrastructure that fuel our client's advanced analytics and machine learning initiatives. As a remote-first organization, strong communication, collaboration, and self-discipline are key. You will work with cutting-edge technologies, guide the technical direction of data engineering projects, and mentor other engineers. This is a significant opportunity to shape the data strategy of a forward-thinking company and drive innovation from anywhere.

Key Responsibilities

• Design, develop, and optimize large-scale batch and real-time data pipelines.
• Build and maintain robust, scalable, and efficient data infrastructure using cloud-native technologies.
• Collaborate with data scientists, analysts, and software engineers to understand data needs and deliver effective solutions.
• Implement best practices for data governance, quality, security, and performance.
• Develop and manage data warehousing solutions and data lakes.
• Evaluate and select appropriate data technologies and tools.
• Automate data processes and workflows using scripting and orchestration tools.
• Troubleshoot and resolve complex data-related issues.
• Mentor and guide junior data engineers and contribute to the team's technical growth.
• Stay abreast of the latest trends and advancements in data engineering and big data technologies.

Requirements

• Master's or Ph.D. in Computer Science, Engineering, or a related quantitative field, or equivalent extensive experience.
• 8+ years of experience in data engineering or a related field.
• Proven experience designing and implementing complex data pipelines using technologies like Spark, Flink, or Kafka.
• Expertise in cloud platforms (AWS, Azure, GCP) and their data services (e.g., S3, Redshift, BigQuery, Snowflake, Databricks).
• Strong proficiency in programming languages such as Python, Scala, or Java.
• Extensive experience with SQL and NoSQL databases.
• Familiarity with data modeling techniques and ETL/ELT processes.
• Experience with containerization (Docker, Kubernetes) and CI/CD practices.
• Excellent leadership, communication, and problem-solving skills, particularly in a remote setting.
• Demonstrated ability to lead technical initiatives and mentor teams.

Benefits

• Highly competitive salary and bonus structure.
• Comprehensive health, dental, and vision benefits.
• Generous allowance for professional development, training, and conferences.
• Fully remote work arrangement with flexible hours.
• Opportunity to work with a talented team on groundbreaking data projects.
• Stock options or equity participation.
• Company-provided home office setup budget.