Remote Principal Data Engineer - Big Data

Placements24; Soweto; Hybrid; ZAR 2,000,000 - 2,800,000; Full time

Benefits offered by this job

Remote-first culture

Competitive compensation package

Learning & development opportunities

Flexible work arrangement

Job summary

Placements24 is seeking a Principal Data Engineer for a fully remote role. You will design, build, and maintain scalable data pipelines and data models supporting analytics, ML, and BI initiatives.

You will lead architectural decisions, ship high-performance data infrastructure on cloud platforms (AWS/Azure/GCP), and mentor junior engineers while ensuring data quality and security.

Qualifications

• Master's or PhD in Computer Science, Engineering or related quantitative field.
• 7+ years in data engineering with big data tech.
• Experience with distributed data processing frameworks like Apache Spark.
• Proficiency in Python, Scala, or Java.
• Strong cloud data warehousing experience (Snowflake, Redshift, BigQuery).
• Solid SQL and data modeling knowledge.
• Experience with real-time streaming tech (Kafka, Kinesis).
• Familiar with containers (Docker, Kubernetes).
• Excellent problem-solving and leadership abilities.

Responsibilities

• Design, build, and maintain scalable data pipelines for large-scale processing.
• Develop and optimize data models and structures for efficient storage and retrieval.
• Implement and manage big data tech and platforms (Spark, Hadoop, Kafka, Snowflake, BigQuery).
• Deploy and manage data solutions on AWS, Azure, or GCP.
• Ensure data quality, integrity, and security across data systems.
• Collaborate with data scientists, analysts, and stakeholders to understand data needs.
• Automate data processes and implement monitoring/alerting.
• Provide technical leadership and mentor junior engineers.
• Evaluate and adopt new data technologies and methodologies.

Skills

Distributed data processing

Python

Scala

Java

Data modeling

SQL

Leadership

Communication

Education

Master's degree in Computer Science

PhD in related quantitative field

Tools

Apache Spark

Kafka

Kubernetes

Docker

Snowflake

BigQuery

Redshift

Job description

About the Role

Our client is a pioneering data analytics company, seeking a Principal Data Engineer for a fully remote position. You will be at the forefront of designing, building, and maintaining scalable, high-performance big data infrastructure. The ideal candidate possesses extensive experience with distributed systems, cloud platforms, and modern data warehousing techniques. You will work with vast datasets, enabling advanced analytics, machine learning, and business intelligence initiatives. This role requires a strategic thinker with a deep technical understanding of the entire data lifecycle, from ingestion and transformation to storage and access. Our client fosters a collaborative, remote-first culture, emphasizing innovation and technical excellence. You will lead architectural decisions, mentor junior engineers, and ensure the reliability and efficiency of our data pipelines, impacting critical decision-making across the organization. This is a unique opportunity to shape the future of data engineering in a flexible, remote work environment.

Key Responsibilities

• Design, build, and maintain scalable and robust data pipelines for large-scale data processing.
• Develop and optimize data models and structures for efficient storage and retrieval.
• Implement and manage big data technologies and platforms (e.g., Spark, Hadoop, Kafka, Snowflake, BigQuery).
• Deploy and manage data solutions on cloud platforms such as AWS, Azure, or GCP.
• Ensure data quality, integrity, and security across all data systems.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements.
• Automate data processes and implement monitoring and alerting systems.
• Provide technical leadership and mentorship to junior data engineers.
• Evaluate and adopt new data technologies and methodologies.

Requirements

• Master's or Ph.D. degree in Computer Science, Engineering, or a related quantitative field.
• Minimum of 7 years of experience in data engineering, with a strong focus on big data technologies.
• Extensive experience with distributed data processing frameworks like Apache Spark.
• Proficiency in programming languages such as Python, Scala, or Java.
• Strong experience with cloud data warehousing solutions (e.g., Snowflake, Redshift, BigQuery).
• Solid understanding of database design, SQL, and data modeling principles.
• Experience with real-time data streaming technologies (e.g., Kafka, Kinesis).
• Familiarity with containerization technologies (e.g., Docker, Kubernetes).
• Excellent problem-solving skills and a passion for building efficient data systems.
• Strong communication and leadership abilities.

Benefits

• Our client offers a highly competitive salary, commensurate with experience, and a comprehensive remote work benefits package. This includes premium health insurance, generous retirement savings plans with company matching, performance bonuses, and ample paid time off.
• We provide opportunities for continuous learning, including access to advanced training, certifications, and conference attendance.
• This fully remote role offers the flexibility to work from your preferred location while contributing to industry-leading data solutions.
• Join our client and drive innovation in the world of big data.