Senior Data Engineer - Big Data Solutions

Placements24; Klerksdorp; On-site; ZAR 900,000 - 1,300,000; Full time

Benefits offered by this job

Competitive salary & benefits

Challenging big data projects in Klerk

Health, dental, and vision insurance

Paid time off & retirement plan

Learning & development support

Job summary

Placements24 in Klerksdorp is seeking a Senior Data Engineer to design and implement robust big data infrastructure. You will build and maintain data pipelines, ensure data quality, and enable analytics and machine learning initiatives.

You will work with large datasets, optimize frameworks such as Spark and Hadoop, and collaborate with data scientists and analysts to deliver reliable data solutions and governance.

Qualifications

• Minimum of 5 years of data engineering experience.
• Proven experience with Spark, Hadoop, Kafka or Flink.
• Strong SQL and Python or Scala.
• Experience with AWS Redshift, Snowflake or BigQuery.
• Solid understanding of data warehousing concepts.

Responsibilities

• Design, build, and maintain scalable ETL/ELT data pipelines using big data tech.
• Develop data warehousing and data lake solutions to support BI.
• Optimize processing frameworks for performance and cost.
• Ensure data quality, integrity, and security across platforms.
• Collaborate with data scientists and analysts to meet data needs.
• Implement data governance to maintain compliance and integrity.

Skills

Spark

Hadoop

Kafka

Flink

SQL

Python

Scala

AWS Redshift

Snowflake

BigQuery

Job description

About the Role

Our client is seeking a highly experienced Senior Data Engineer to lead the design and implementation of their big data infrastructure. Based in our Klerksdorp office, this role is crucial for building and maintaining robust data pipelines, ensuring data quality, and enabling advanced analytics and machine learning initiatives. You will work with large-scale datasets, optimize data processing frameworks, and collaborate with data scientists and analysts to deliver reliable data solutions. This is a fantastic opportunity for a seasoned data professional to shape the future of data at the organization.

Key Responsibilities

• Design, build, and maintain scalable and efficient ETL/ELT data pipelines using big data technologies.
• Develop and manage data warehousing solutions and data lakes to support business intelligence and analytics needs.
• Optimize data processing frameworks (e.g., Spark, Hadoop) for performance and cost-efficiency.
• Ensure data quality, integrity, and security across all data platforms.
• Collaborate with data scientists and analysts to understand data requirements and provide accessible, high-quality data.
• Implement data governance policies and procedures to maintain compliance and data integrity.

Requirements

• Minimum of 5 years of experience in data engineering or a related field.
• Proven experience with big data technologies such as Apache Spark, Hadoop, Kafka, or Flink.
• Strong proficiency in SQL and programming languages like Python or Scala.
• Experience with cloud-based data platforms (e.g., AWS Redshift, Snowflake, BigQuery).
• Deep understanding of data warehousing concepts and best practices.
• Excellent problem-solving skills and ability to work independently and in a team.

Benefits

• Competitive salary and comprehensive benefits package.
• Opportunity to work on challenging big data projects in Klerksdorp .
• Health, dental, and vision insurance.
• Generous paid time off and retirement savings plan.
• Continuous learning opportunities and professional development support.