Senior Data Engineer (Big Data)

Placements24

Soweto

Hybrid

ZAR 900,000 - 1,350,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Competitive salary
Work from home
Flexible working hours
Health insurance stipend
Professional development budget
Paid time off
Remote collaboration tools

Job summary

Placements24 is searching for a Senior Data Engineer to join a fully remote team. You will design, build, and maintain scalable data pipelines and infrastructure to support analytical and ML initiatives. Expect collaboration with data scientists and engineers across distributed environments.

The role emphasizes data governance, quality, and scalable architectures using Spark, Hadoop and cloud platforms, with a focus on reliability and security.

Qualifications

  • Advanced knowledge of Spark/Hadoop for large-scale data pipelines.
  • Proficient in Python/Scala/Java and SQL.
  • Experience with cloud data platforms and data warehousing concepts.

Responsibilities

  • Design, build, and optimize robust and scalable big data pipelines and ETL processes.
  • Develop and manage data warehousing solutions and data lakes using Hadoop, Spark, and cloud platforms.
  • Implement data governance and data quality best practices.
  • Collaborate with data scientists and analysts to understand data needs and deliver solutions.
  • Develop and maintain data models for various analytical purposes.
  • Ensure performance, reliability, and security of data infrastructure.
  • Automate data processes and implement monitoring and alerting systems.
  • Stay current with emerging big data technologies and trends.
  • Troubleshoot data-related issues across the data lifecycle.
  • Contribute to data platform architecture decisions.

Skills

Python
Scala
Java
Spark
Hadoop
AWS EMR
Azure Databricks
Google Cloud Dataflow
SQL
NoSQL
Airflow
Data Modeling
Data Warehousing

Education

Bachelor's or Master's in CS/Engineering

Tools

HDFS
MapReduce
ETL Tools
NoSQL Databases
Apache Airflow

Job description

About the Role

Our client, a prominent data analytics consultancy, is seeking a Senior Data Engineer with extensive experience in big data technologies to join their fully remote team. This role is essential for designing, building, and maintaining scalable data pipelines and infrastructure that support complex analytical and machine learning initiatives. You will work with large, diverse datasets, ensuring data quality, reliability, and accessibility for data scientists and analysts. The ideal candidate possesses deep technical expertise in distributed systems, data warehousing, and ETL processes, and thrives in a collaborative, remote environment. This is a fantastic opportunity to work on challenging big data projects and influence data strategy from anywhere.

Key Responsibilities
  • Design, build, and optimize robust and scalable big data pipelines and ETL processes.
  • Develop and manage data warehousing solutions and data lakes using technologies like Hadoop, Spark, and cloud-based data platforms.
  • Implement data governance and data quality best practices.
  • Collaborate with data scientists and analysts to understand their data needs and provide efficient data solutions.
  • Develop and maintain data models for various analytical purposes.
  • Ensure the performance, reliability, and security of data infrastructure.
  • Automate data processes and implement monitoring and alerting systems.
  • Stay current with emerging big data technologies and trends.
  • Troubleshoot and resolve data-related issues across the entire data lifecycle.
  • Contribute to architectural decisions regarding data platform design and implementation.
Requirements
  • Bachelor's or Master's degree in Computer Science, Engineering, Statistics, or a related quantitative field.
  • Minimum of 5 years of experience in data engineering, with a strong focus on big data technologies.
  • Proficiency in programming languages such as Python, Scala, or Java.
  • Hands-on experience with distributed computing frameworks like Apache Spark and Hadoop (HDFS, MapReduce).
  • Experience with cloud data platforms (e.g., AWS EMR, Azure Databricks, Google Cloud Dataflow).
  • Strong knowledge of SQL and experience with various database technologies (relational and NoSQL).
  • Familiarity with data warehousing concepts and ETL/ELT tools.
  • Experience with workflow management tools (e.g., Apache Airflow).
  • Understanding of data modeling techniques and principles.
  • Excellent problem-solving skills and ability to work with complex, large-scale datasets.
  • Strong communication skills and ability to collaborate effectively in a remote team environment.
Benefits
  • This is a fully remote position offering excellent flexibility.
  • Our client provides a highly competitive salary, regular performance reviews, and potential for bonuses.
  • Comprehensive health, dental, and vision insurance coverage are included.
  • A generous retirement savings plan with company contributions is available.
  • We support continuous learning through online courses, certifications, and virtual conferences.
  • Employees receive ample paid time off.
  • The company culture fosters collaboration, innovation, and values employee well-being, even in a distributed setting.
  • Opportunities for professional development are abundant, and the company invests in tools and resources to ensure remote employees succeed.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer (Big Data Pipelines)
Senior Data Engineer (Big Data Pipelines)

Placements24 • Noord-Kaap

Hybrid
ZAR 900,000 - 1,500,000
Remote Senior Data Engineer - Scalable Big Data Pipelines
Remote Senior Data Engineer - Scalable Big Data Pipelines

Placements24 • East London

Remote
ZAR 1,200,000 - 2,000,000
Highly competitive compensation
Fully remote work with flexible hours
Professional development budget
+5
Lead Data Engineer (Big Data Specialist)
Lead Data Engineer (Big Data Specialist)

Placements24 • Gauteng

Hybrid
ZAR 1,996,000 - 2,995,000
Competitive compensation
Full remote work
Learning and development opportunities
+2
Senior Data Engineer (Big Data & Cloud) - Remote
Senior Data Engineer (Big Data & Cloud) - Remote

Placements24 • Upington

Hybrid
ZAR 1,000,000 - 1,300,000
Competitive salary
Performance bonuses
Health & wellness benefits
+2
Principal Data Engineer
Principal Data Engineer

Placements24 • LegKraal Gate

Hybrid
ZAR 1,200,000 - 2,000,000
Health insurance
Remote work with flexible hours
Professional development budget
+1
Senior Data Engineer – Cloud Data & Real-Time Analytics
Senior Data Engineer – Cloud Data & Real-Time Analytics

Digital Outsource Services • Cape Town

Remote
ZAR 900,000 - 1,500,000
Competitive salary and bonuses
Fully remote work arrangement
Health, dental, and vision insurance
+1
Senior Data Scientist - Big Data Analytics
Senior Data Scientist - Big Data Analytics

Placements24 • Midrand

Hybrid
ZAR 1,976,000 - 2,964,000
Fully remote
Health, dental, and vision insurance
Competitive salary
+5
Principal Data Engineer
Principal Data Engineer

Placements24 • Sandton

Hybrid
ZAR 1,200,000 - 1,800,000
Bonus structure
Health benefits
Professional development
+3
Senior Data Engineer - Cloud Platforms
Senior Data Engineer - Cloud Platforms

Placements24 • Upington

Hybrid
ZAR 900,000 - 1,500,000
Competitive salary
Remote benefits package
Home office setup budget
+2
Remote Software Developer - Data Platforms
Remote Software Developer - Data Platforms

Placements24 • Benoni

Hybrid
ZAR 1,482,000 - 2,470,000
Full remote work flexibility
Health, dental, and vision insurance
Generous PTO and holidays
+4