Remote Senior Data Engineer - Big Data Platforms

Placements24; East London; Hybrid; ZAR 900,000 - 1,500,000; Full time

Benefits offered by this job

Remote work

Stock options

Health insurance

Professional development budget

Home office stipend

Job summary

Placements24 is seeking a Remote Senior Data Engineer for a cutting-edge data analytics company operating with a fully remote workforce. The role focuses on designing, building, and maintaining scalable data pipelines and infrastructure to support advanced analytics and ML initiatives.

You will collaborate with data scientists and analysts, work with cloud platforms (AWS/Azure/GCP), SQL/NoSQL, Spark/Flink/Hadoop, and ETL/ELT processes in a remote, flexible environment.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.
• 5+ years of experience as a Data Engineer, with a strong focus on big data technologies.
• Proven experience with cloud platforms such as AWS (S3, Redshift, EMR), Azure (Data Lake, Synapse), or GCP (BigQuery, Dataflow).
• Proficiency in SQL and experience with NoSQL databases.
• Hands-on experience with distributed data processing frameworks like Apache Spark, Hadoop, or Flink.
• Expertise in programming languages such as Python or Scala for data engineering tasks.
• Experience with data warehousing concepts and tools.
• Strong understanding of ETL/ELT processes and data pipeline orchestration tools (e.g., Airflow, Luigi).
• Excellent problem-solving, analytical, and communication skills.
• Ability to work independently and effectively in a remote team environment.

Responsibilities

• Design, construct, install, test, and maintain highly scalable data management systems and data pipelines.
• Develop and implement ETL/ELT processes to ingest, transform, and load data from various sources.
• Build and optimize data architectures and data warehouses for performance and efficiency.
• Work with large datasets, ensuring data quality, integrity, and availability.
• Implement data modeling techniques to support business intelligence and analytical requirements.
• Collaborate with data scientists and analysts to understand their data needs and provide solutions.
• Optimize data query performance and database performance.
• Ensure adherence to data governance, security, and compliance standards.
• Stay current with emerging big data technologies and best practices.
• Troubleshoot and resolve data-related issues in production environments.

Skills

Big data

SQL

Python

Scala

Communication skills

Independent work

Education

Bachelor's or Master's degree in Computer Science/Engineering

Tools

AWS

Azure

GCP

Airflow

Luigi

Spark

Hadoop

Flink

Job description

About the Role

Our client, a cutting-edge data analytics company operating with a fully remote workforce, is seeking a highly skilled Remote Senior Data Engineer to join their expanding team. This position is pivotal in designing, building, and maintaining scalable data pipelines and infrastructure that support the company's advanced analytics and machine learning initiatives. The ideal candidate possesses extensive experience with big data technologies, cloud platforms, and data warehousing solutions. You will collaborate with data scientists, analysts, and software engineers to ensure the efficient and reliable flow of data across the organization. This is an exceptional opportunity to work on complex data challenges, contribute to architectural decisions, and leverage the latest technologies in a fully remote, flexible, and innovative environment. The role is based conceptually around East London but is entirely remote.

Key Responsibilities

• Design, construct, install, test, and maintain highly scalable data management systems and data pipelines.
• Develop and implement ETL/ELT processes to ingest, transform, and load data from various sources.
• Build and optimize data architectures and data warehouses for performance and efficiency.
• Work with large datasets, ensuring data quality, integrity, and availability.
• Implement data modeling techniques to support business intelligence and analytical requirements.
• Collaborate with data scientists and analysts to understand their data needs and provide solutions.
• Optimize data query performance and database performance.
• Ensure adherence to data governance, security, and compliance standards.
• Stay current with emerging big data technologies and best practices.
• Troubleshoot and resolve data-related issues in production environments.

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.
• 5+ years of experience as a Data Engineer, with a strong focus on big data technologies.
• Proven experience with cloud platforms such as AWS (S3, Redshift, EMR), Azure (Data Lake, Synapse), or GCP (BigQuery, Dataflow).
• Proficiency in SQL and experience with NoSQL databases.
• Hands-on experience with distributed data processing frameworks like Apache Spark, Hadoop, or Flink.
• Expertise in programming languages such as Python or Scala for data engineering tasks.
• Experience with data warehousing concepts and tools.
• Strong understanding of ETL/ELT processes and data pipeline orchestration tools (e.g., Airflow, Luigi).
• Excellent problem-solving, analytical, and communication skills.
• Ability to work independently and effectively in a remote team environment.

Benefits

• Fully remote work arrangement with flexible hours.
• Highly competitive salary and performance bonus.
• Comprehensive health, dental, and vision insurance.
• Stock options plan.
• Generous budget for professional development, including conferences and training.
• Opportunity to work with cutting-edge big data technologies.
• Collaborative and innovative company culture.
• Home office stipend and necessary equipment provided.