Senior Big Data Engineer: Etl, Data Lakes & Analytics

Placements24; Klerksdorp; Remote; ZAR 900,000 - 1,500,000; Full time

Benefits offered by this job

Health and wellness benefits

Remote work flexibility

Generous paid time off

Professional development support

Job summary

Placements24 is seeking a Senior Data Engineer in Klerksdorp to design and implement scalable data infrastructure for analytics and ML. You will build and maintain robust ETL/ELT pipelines, data lakes and warehouses, and partner with data scientists to unlock actionable insights.

The role requires 5+ years in data engineering, deep knowledge of Spark/Hadoop/Kafka, SQL and Python/Scala, and experience with cloud data services. Remote work options are available with a strong benefits package.

Qualifications

• 5+ years of data engineering experience with big data tech.
• Bachelor's or Master's degree in CS/Engineering or related field.
• Proficiency in SQL and Python/Scala.
• Experience with Spark, Hadoop and Kafka.
• Experience with cloud data platforms (AWS/Azure/GCP).
• Knowledge of data warehousing concepts.
• Strong analytical and collaboration skills.

Responsibilities

• Design, build, and maintain ETL/ELT pipelines.
• Develop data warehouses and data lakes to support analytics.
• Optimize data processing using Spark/Hadoop.
• Ensure data quality, integrity, and security.
• Collaborate with data scientists and analysts.
• Implement data governance and monitoring.

Skills

Big data

SQL

Python/Scala

Spark

Hadoop

Data warehousing

Data governance

Education

Bachelor's degree

Master's degree

Tools

Airflow

Kafka

AWS

Job description

Our client is seeking a highly experienced Senior Data Engineer to lead the design and implementation of their big data infrastructure. Based in our Klerksdorp office, this role is crucial for building and maintaining robust data pipelines, ensuring data quality, and enabling advanced analytics and machine learning initiatives. You will work with large-scale datasets, optimize data processing frameworks, and collaborate with data scientists and analysts to deliver reliable data solutions. This is a fantastic opportunity for a seasoned data professional to shape the future of data at the organization.

Key Responsibilities

• Design, build, and maintain scalable and efficient ETL/ELT data pipelines using big data technologies.
• Develop and manage data warehousing solutions and data lakes to support business intelligence and analytics needs.
• Optimize data processing frameworks (e.g., Spark, Hadoop) for performance and cost-efficiency.
• Ensure data quality, integrity, and security across all data platforms.
• Collaborate with data scientists and analysts to understand data requirements and provide accessible, high-quality data.
• Implement data governance policies and procedures to maintain compliance and data integrity.

Requirements

• Minimum of 5 years of experience in data engineering or a related field.
• Proven experience with big data technologies such as Apache Spark, Hadoop, Kafka, or Flink.
• Strong proficiency in SQL and programming languages like Python or Scala.
• Experience with cloud-based data platforms (e.g., AWS Redshift, Snowflake, BigQuery).
• Deep understanding of data warehousing concepts and best practices.
• Excellent problem-solving skills and ability to work independently and in a team.

Benefits

• Competitive salary and comprehensive benefits package.
• Opportunity to work on challenging big data projects in Klerksdorp.
• Health, dental, and vision insurance.
• Generous paid time off and retirement savings plan.
• Continuous learning opportunities and professional development support.

Our client is seeking an experienced Senior Data Engineer to build and maintain scalable data infrastructure that supports advanced analytics and AI initiatives in Klerksdorp. You will be responsible for designing, developing, and optimizing data pipelines, ensuring the availability and quality of data for machine learning models and business intelligence. This role is critical for empowering data scientists and analysts with the data they need to drive insights and innovation. The ideal candidate has a strong background in big data technologies and a passion for enabling data-driven decision-making.

Key Responsibilities

• Design, build, and maintain robust, scalable ETL/ELT pipelines for large datasets.
• Develop and manage data warehousing solutions and data lakes using big data technologies.
• Ensure data quality, integrity, and security across all data systems.
• Collaborate with data scientists and analysts to understand data requirements and provide necessary data access.
• Optimize data infrastructure for performance, cost-efficiency, and reliability.
• Implement data governance best practices and monitoring solutions.

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 5+ years of experience in data engineering, with a focus on big data technologies (e.g., Spark, Hadoop, Kafka).
• Proficiency in SQL and programming languages like Python or Scala.
• Experience with cloud data platforms (AWS, Azure, GCP) and their data services.
• Familiarity with data modeling, database design, and data warehousing concepts.
• Strong understanding of data pipeline orchestration tools (e.g., Airflow) and MLOps concepts is a plus.

Benefits

• Attractive salary and bonus structure.
• Fully remote position, providing exceptional work-life balance.
• Comprehensive health and wellness benefits package.
• Opportunities for continuous learning and career advancement in a growing field.
• Be part of a collaborative and innovative team focused on cutting-edge data solutions.

Our client is seeking a highly analytical and experienced Senior Data Engineer to join their innovative team, working remotely. You will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure that support advanced analytics and machine learning initiatives. This role is crucial for enabling data-driven decision-making across the organization. You will collaborate with data scientists, analysts, and business stakeholders to understand data needs and deliver robust data solutions. Join us from anywhere in South Africa and help shape the future of our data capabilities.

Key Responsibilities

• Design, build, and optimize robust and scalable data pipelines for batch and real-time data processing.
• Develop and maintain data warehousing solutions and data lake architectures.
• Ensure data quality, integrity, and accessibility for analytics and reporting purposes.
• Collaborate with data scientists and analysts to support their data requirements.
• Implement and manage data governance and security best practices.
• Monitor and troubleshoot data infrastructure, ensuring optimal performance and reliability.

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, Statistics, or a related quantitative field.
• 5+ years of experience in data engineering or a similar role.
• Strong proficiency in SQL and experience with data warehousing concepts.
• Hands-on experience with big data technologies (e.g., Spark, Hadoop) and cloud platforms (AWS, Azure, GCP).
• Proficiency in at least one programming language such as Python or Scala.
• Excellent analytical and problem-solving skills, with strong attention to detail.

Benefits

• Competitive salary and performance-based bonuses.
• Comprehensive health and wellness benefits.
• Generous opportunities for professional development and continuous learning.
• Flexible remote work arrangements.
• Data-centric culture that values innovation and collaboration.