Senior Data Engineer - Cloud Platforms

Placements24; Upington; Hybrid; ZAR 900,000 - 1,500,000; Full time

Benefits offered by this job

Competitive salary

Remote benefits package

Home office setup budget

Flexible working hours

Energy sector projects

Job summary

Placements24 is seeking a Senior Data Engineer for a fully remote, cloud-native role in the energy sector. You will design and optimize scalable data pipelines across AWS, Azure, and GCP, enabling analytics and machine learning on diverse datasets from exploration to operations.

The role requires 7+ years of cloud data engineering experience, strong SQL, and Python/Scala, with a focus on data governance and performance optimization. Remote work with flexible hours is offered.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related quantitative field.
• 7+ years of data engineering experience with cloud-based data solutions.
• Strong SQL and Python/Scala development skills.

Responsibilities

• Design, build, and maintain scalable data pipelines on cloud platforms (AWS, Azure, GCP).
• Develop ETL/ELT processes for ingesting, transforming, and loading data into data warehouses and data lakes.
• Optimize data storage and processing performance for large-scale datasets.
• Implement data governance and security best practices in cloud environments.
• Collaborate with data scientists and analysts to ensure data is readily available and meets their needs.
• Evaluate and recommend new technologies and tools to enhance the data infrastructure.

Skills

SQL

Python

Scala

Education

Bachelor's degree in Computer Science, Engineering, or related field

Tools

AWS

Azure

GCP

BigQuery

Redshift

S3

Glue

Data Lake

Synapse

Dataflow

Job description

About the Role

Our client is seeking a highly skilled and experienced Senior Data Engineer to join their fully remote, cloud-native team, focusing on building and optimizing robust data pipelines for the energy sector. This role is pivotal in architecting, developing, and maintaining scalable data infrastructure on major cloud platforms, enabling advanced analytics and machine learning initiatives. You will work with diverse datasets from exploration, production, and operational systems to ensure data quality, accessibility, and performance. This is a fantastic remote opportunity for a data engineering expert passionate about leveraging cloud technologies to drive data-driven decision-making in the complex and dynamic oil and gas industry.

Key Responsibilities

• Design, build, and maintain scalable and reliable data pipelines on cloud platforms (AWS, Azure, GCP).
• Develop ETL/ELT processes for ingesting, transforming, and loading data from various sources into data warehouses and data lakes.
• Optimize data storage and processing performance for large-scale datasets.
• Implement data governance and security best practices in cloud environments.
• Collaborate with data scientists and analysts to ensure data is readily available and meets their needs.
• Evaluate and recommend new technologies and tools to enhance the data infrastructure.

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related quantitative field.
• Over 7 years of experience in data engineering, with a strong emphasis on cloud-based data solutions.
• Extensive experience with cloud platforms such as AWS (S3, Redshift, Glue), Azure (Data Lake, Synapse), or GCP (BigQuery, Dataflow).
• Proficiency in SQL and at least one programming language like Python or Scala.
• Experience with big data technologies (e.g., Spark, Hadoop) and data warehousing concepts.
• Strong understanding of data modeling, database design, and data governance principles.
• Excellent problem-solving skills and ability to work effectively in a remote team.

Benefits

• Highly competitive salary and comprehensive remote benefits package.
• Generous home office setup budget and ongoing technology support.
• Flexible working hours and the ability to work from anywhere.
• Opportunities to work on transformative data projects in the energy sector.
• A collaborative and forward-thinking remote work culture.