Principal Data Engineer

Placements24; LegKraal Gate; Hybrid; ZAR 1,200,000 - 2,000,000; Full time

Benefits offered by this job

Health insurance

Remote work with flexible hours

Professional development budget

Company-provided hardware

Job summary

Placements24 is seeking a Principal Data Engineer to design, build, and maintain scalable data infrastructure in a fully remote role. You will lead data pipelines, architect data lakes and warehouses on cloud platforms, and mentor engineers while collaborating with data scientists and software teams to deliver reliable, secure analytics solutions.

The ideal candidate has 8+ years in data engineering, strong SQL/NoSQL expertise, and hands-on experience with Spark, Hadoop, Kafka, and modern cloud

Qualifications

• Master's degree in Computer Science, Engineering, or related field or equivalent extensive experience.
• 8+ years of experience in data engineering or related fields.
• Proven experience designing and implementing large-scale data solutions on cloud platforms (AWS, Azure, or GCP).
• Expertise in Spark, Hadoop, Kafka, and other big data technologies.
• Strong proficiency in SQL and NoSQL databases.
• Experience with data warehousing concepts and tools (e.g., Snowflake, Redshift, BigQuery).
• Proficiency in Python, Scala, or Java for data processing.
• Experience with data modeling, governance, and security best practices.
• Excellent problem-solving, communication, and leadership skills.

Responsibilities

• Design, develop, and maintain scalable data pipelines using ETL/ELT processes.
• Architect data lakes, data warehouses, and data storage solutions on cloud platforms (AWS, Azure, GCP).
• Optimize data infrastructure for performance, scalability, and cost-effectiveness.
• Develop data models to support analytics and reporting.
• Ensure data quality, integrity, and security across systems.
• Collaborate with data scientists, analysts, and engineers to fulfill data requirements.
• Implement and manage data processing frameworks (e.g., Spark, Flink).
• Automate deployment with infrastructure as code and CI/CD practices.
• Mentor junior data engineers and promote best practices.

Skills

ETL/ELT

Data pipelines

Distributed systems

SQL

NoSQL

Python

Scala

Java

Education

Master's degree in Computer Science or Engineering

Tools

Spark

Hadoop

Kafka

Snowflake

Redshift

BigQuery

Terraform

Ansible

Job description

About the Role

Our client, a leader in cloud-based data solutions, is seeking a highly experienced Principal Data Engineer to design, build, and maintain scalable and robust data infrastructure. This fully remote position is crucial for developing and optimizing data pipelines, ensuring the availability and integrity of data for analytics and machine learning initiatives. You will work with cutting-edge big data technologies, architect data lakes and warehouses, and collaborate closely with data scientists, analysts, and software engineers. The ideal candidate possesses deep expertise in distributed systems, data modeling, ETL processes, and cloud data platforms. This is a unique opportunity to lead data engineering efforts, mentor other engineers, and drive innovation in data architecture from anywhere in the world. We are looking for a visionary engineer who can translate complex data requirements into efficient, high-performance data solutions.

Key Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines using ETL/ELT processes.
• Architect and implement data lakes, data warehouses, and other data storage solutions on cloud platforms (AWS, Azure, GCP).
• Optimize data infrastructure for performance, scalability, and cost-effectiveness.
• Develop and manage data models that support various analytical and reporting needs.
• Ensure data quality, integrity, and security across all data systems.
• Collaborate with data scientists, analysts, and application developers to understand data requirements and provide data solutions.
• Implement and manage data processing frameworks (e.g., Spark, Flink).
• Automate data infrastructure deployment and management using infrastructure as code and CI/CD practices.
• Mentor junior data engineers and promote best practices in data engineering.
• Stay current with emerging data technologies and trends.

Requirements

• Master's degree in Computer Science, Engineering, or a related quantitative field, or equivalent extensive experience.
• 8+ years of experience in data engineering or related fields.
• Proven experience designing and implementing large-scale data solutions on cloud platforms (AWS, Azure, or GCP).
• Expertise in big data technologies such as Apache Spark, Hadoop, Kafka, etc.
• Strong proficiency in SQL and NoSQL databases.
• Experience with data warehousing concepts and tools (e.g., Snowflake, Redshift, BigQuery).
• Proficiency in programming languages like Python, Scala, or Java for data processing.
• Experience with data modeling and schema design.
• Familiarity with data governance and data security best practices.
• Excellent problem-solving, communication, and leadership skills.
• Experience with CI/CD and infrastructure as code (e.g., Terraform, Ansible) is a plus.

Benefits

• Top-tier compensation package including base salary and significant performance bonuses.
• Comprehensive health, dental, and vision insurance coverage.
• Generous budget for professional development, conferences, and training.
• Fully remote work arrangement with flexible hours.
• Company-provided high-performance computing equipment.
• Unlimited paid time off and company holidays.
• Opportunity to shape the data architecture of a forward-thinking organization.
• Collaborative and supportive global team environment.
• Regular virtual team-building events and knowledge-sharing sessions.
• 401(k) or equivalent retirement savings plan with company match.