Senior Data Engineer

Placements24; Mossel Bay; Hybrid; ZAR 900,000 - 1,300,000; Full time

Job summary

Placements24 is seeking a highly skilled Senior Data Engineer to join their fully remote data team. You will design, build, and maintain scalable data pipelines and infrastructure powering analytics and ML initiatives for a global business.

You will work with large datasets, collaborate with data scientists, and implement data quality checks, data modeling, and cloud-based data warehousing on AWS/GCP/Azure. This role offers a flexible remote setup and room to influence data-driven decisions.

Qualifications

• Minimum of 6 years of experience in data engineering or related field.
• Strong SQL and at least one programming language (Python or Scala).
• Hands-on experience with big data technologies (Spark, Hadoop).
• Experience designing and implementing data solutions on cloud platforms (AWS, Azure, GCP).
• Solid understanding of data modeling, data warehousing concepts, and database design.

Responsibilities

• Design, develop, and maintain scalable and efficient ETL/ELT data pipelines using tools like Apache Spark, Airflow, or similar.
• Build and optimize data warehousing solutions and data lakes on cloud platforms (e.g., AWS Redshift, Google BigQuery, Azure Synapse).
• Implement data quality checks and monitoring to ensure the accuracy and reliability of data.
• Collaborate with data scientists and analysts to understand their data needs and provide solutions.
• Optimize database performance and data access for various analytical workloads.
• Develop and maintain data infrastructure, ensuring security, scalability, and cost-effectiveness.

Skills

SQL

Python

Scala

Remote teamwork

Tools

Apache Spark

Airflow

Hadoop ecosystem

AWS

Azure

GCP

Job description

About the Role

Our client is looking for a highly skilled Senior Data Engineer to join their fully remote data team. This role is pivotal in designing, building, and maintaining robust data pipelines and infrastructure that power our analytics and machine learning initiatives. You will work with large, complex datasets, transforming raw data into actionable insights that drive business decisions. This is a fantastic opportunity for a seasoned data professional to contribute to a data-driven culture in a flexible, remote environment, working with modern data technologies and collaborating with a talented group of data scientists and analysts.

Key Responsibilities

• Design, develop, and maintain scalable and efficient ETL/ELT data pipelines using tools like Apache Spark, Airflow, or similar.


• Build and optimize data warehousing solutions and data lakes on cloud platforms (e.g., AWS Redshift, Google BigQuery, Azure Synapse).


• Implement data quality checks and monitoring to ensure the accuracy and reliability of data.


• Collaborate with data scientists and analysts to understand their data needs and provide solutions.


• Optimize database performance and data access for various analytical workloads.


• Develop and maintain data infrastructure, ensuring security, scalability, and cost-effectiveness.

Requirements

• Minimum of 6 years of experience in data engineering or a related field.


• Strong proficiency in SQL and experience with at least one programming language such as Python or Scala.


• Hands-on experience with big data technologies (e.g., Spark, Hadoop ecosystem).


• Experience designing and implementing data solutions on cloud platforms (AWS, Azure, GCP).


• Solid understanding of data modeling, data warehousing concepts, and database design.


• Excellent problem-solving abilities and experience working effectively in a remote team setting.

Benefits

• Highly competitive annual salary with performance-based bonuses.


• Comprehensive health, dental, and vision insurance.


• Generous paid time off and flexible working arrangements.


• Budget for professional development, conferences, and certifications.


• A collaborative, inclusive, and fully remote work environment.