Remote Senior Data Engineer - Scalable Big Data Pipelines

Placements24; East London; Remoto; ZAR 1.200.000 - 2.000.000; Tempo pieno

Vantaggi offerti da questo lavoro

Highly competitive compensation

Fully remote work with flexible hours

Professional development budget

Home office stipend

Company-provided equipment

Equity or stock option plan

Retirement savings plan with company-m

Contributed by Placements24

Descrizione del lavoro

Placements24 is seeking a Principal Data Engineer to join a fully remote, world-class data engineering team. This senior role designs, builds, and maintains scalable data pipelines, data warehouses, and data lakes for analytics, ML, and BI with a focus on quality, governance, and performance.

The ideal candidate has 8+ years of data engineering experience, mastery of Spark, Kafka, Flink, SQL, Python/Scala/Java, and cloud platforms (AWS/Azure/GCP).

Competenze

• Master’s or PhD in Computer Science, Engineering, or related field or equivalent experience.
• 8+ years of data engineering, data warehousing, or related roles.
• Expertise in designing and building large-scale data pipelines with Spark, Kafka, Flink.
• SQL proficiency and experience with PostgreSQL, Snowflake, Redshift, BigQuery.
• Strong programming in Python, Scala, or Java.
• Experience with cloud data platforms (AWS, Azure, GCP).
• Deep understanding of data modeling, warehousing concepts, and governance.
• Containerization (Docker) and orchestration (Kubernetes) desirable.
• Excellent problem-solving and remote collaboration skills.

Mansioni

• Design, develop, and maintain scalable data pipelines and ETL/ELT processes.
• Architect data warehousing solutions and data lakes.
• Optimize data retrieval, storage, and processing for performance and cost.
• Ensure data quality, integrity, and security across platforms.
• Collaborate with data scientists, analysts and stakeholders to define requirements.
• Develop and implement data modeling strategies.
• Evaluate and integrate new data technologies and tools.
• Provide technical leadership and mentorship to junior data engineers.
• Troubleshoot and resolve complex data-related issues.
• Contribute to overall data strategy and architecture.

Conoscenze

Data modeling

Data governance

Communication

Remote collaboration

Formazione

Master's or PhD in Computer Science/Engineering or related field

Equivalent experience

Strumenti

Spark

Kafka

Flink

PostgreSQL

Snowflake

Redshift

BigQuery

Python

Scala

Java

AWS

Azure

GCP

Docker

Kubernetes

Descrizione del lavoro

About the Role

Our client is seeking an exceptional Principal Data Engineer to join their fully remote, world-class data engineering team. This senior role is pivotal in designing, building, and maintaining the company's data infrastructure, ensuring the availability, reliability, and performance of data pipelines and systems. You will tackle complex data challenges, architecting scalable solutions that support advanced analytics, machine learning, and business intelligence initiatives. As a remote-first organization, we value collaboration, autonomy, and innovation. The ideal candidate is a seasoned expert in big data technologies, data warehousing, and ETL/ELT processes, with a passion for data quality and governance. This is a unique opportunity to shape the future of data at a leading company.

Key Responsibilities

• Design, develop, and maintain scalable and robust data pipelines and ETL/ELT processes.
• Architect and implement data warehousing solutions and data lakes.
• Optimize data retrieval, storage, and processing for performance and cost efficiency.
• Ensure data quality, integrity, and security across all data platforms.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and provide solutions.
• Develop and implement data modeling strategies.
• Evaluate and integrate new data technologies and tools.
• Provide technical leadership and mentorship to junior data engineers.
• Troubleshoot and resolve complex data-related issues.
• Contribute to the overall data strategy and architecture of the organization.

Requirements

• Master's or PhD in Computer Science, Engineering, or a related quantitative field, or equivalent experience.
• 8+ years of experience in data engineering, data warehousing, or related roles.
• Expertise in designing and building large-scale data pipelines using technologies like Spark, Kafka, Flink.
• Proficiency in SQL and experience with database systems such as PostgreSQL, Snowflake, Redshift, BigQuery.
• Strong programming skills in Python, Scala, or Java.
• Experience with cloud data platforms (AWS, Azure, GCP) and their services.
• Deep understanding of data modeling, data warehousing concepts, and data governance.
• Experience with containerization (Docker) and orchestration (Kubernetes) is desirable.
• Excellent problem-solving abilities and a proactive approach to challenges.
• Strong communication and collaboration skills, especially in a remote setting.

Benefits

• Highly competitive salary and significant bonus opportunities.
• Comprehensive health, dental, and vision insurance.
• Generous equity or stock option plan.
• Significant professional development budget for training, conferences, and certifications.
• Fully remote work arrangement with flexible hours.
• Home office stipend and equipment allowance.
• Opportunity to lead and innovate in the data space.
• Collaborative and inclusive remote team environment.
• Company-provided laptop and software.
• Retirement savings plan with company match.