Senior Data Engineer (Big Data & Analytics)

Placements24; United States; Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

Fully remote work

Home office setup allowance

Professional development & conferences

Generous PTO

Retirement plan

Job summary

Placements24 is seeking a Senior Data Engineer to design, build, and maintain scalable data pipelines and data warehouses for analytics in a remote-first setup. The role emphasizes handling massive datasets, leveraging Spark, Hadoop, and cloud platforms, while collaborating with data scientists and business stakeholders to deliver actionable insights.

You will drive data quality, ETL/ELT processes, and data modeling best practices, with a strong focus on security and compliance in a distributed

Qualifications

• Bachelor's or Master’s degree in Computer Science, Data Science, Engineering, or related quantitative field.
• Minimum of 5 years of experience in data engineering with a focus on big data technologies.
• Proficiency in Python, Scala, or Java.
• Hands-on experience with Spark, Hadoop ecosystem (HDFS, MapReduce, Hive).
• Experience with cloud data platforms (AWS, Azure, GCP).
• Strong SQL skills; experience with relational and NoSQL databases.
• Knowledge of data warehousing concepts and ETL/ELT tools.
• Understanding of data modeling techniques and best practices.
• Excellent problem-solving and ability to work independently; strong communication for cross-team collaboration.

Responsibilities

• Design, build, and optimize scalable data pipelines for large data volumes.
• Create data warehousing and data lake solutions using big data technologies.
• Implement data quality frameworks to ensure accuracy and reliability.
• Develop ETL/ELT processes to support BI and analytics.
• Collaborate with data scientists, analysts, and stakeholders to understand data needs.
• Monitor and troubleshoot data pipeline performance and infrastructure issues.
• Stay abreast of emerging big data technologies and best practices.
• Ensure data security and regulatory compliance.

Skills

Python

Scala

Java

Spark

Hadoop

SQL

NoSQL

Data modeling

Problem solving

Communication

Education

Bachelor's degree

Master's degree

Tools

AWS EMR

S3

Redshift

Azure Data Lake

Databricks

GCP BigQuery

HDFS

MapReduce

Hive

Kafka

Job description

About the Role Our client, a leading telecommunications innovator in Klerksdorp, North West , is seeking a Senior Data Engineer specializing in Big Data and Analytics. This is a remote-first position, offering the flexibility to work from anywhere within South Africa . You will be responsible for designing, building, and maintaining robust data pipelines and infrastructure that support our advanced analytics capabilities. This role is critical in transforming raw data into actionable insights, enabling data-driven decision-making across the organization. You will work with massive datasets, leveraging cutting-edge big data technologies and cloud platforms. The ideal candidate possesses a strong technical background in data engineering, a passion for solving complex data challenges, and the ability to collaborate effectively in a remote team environment. This is an exciting opportunity to shape the data landscape of a dynamic telecommunications company and contribute to its strategic growth through powerful data insights. We foster an environment of continuous learning and innovation for our remote workforce.

Key Responsibilities

• Design, develop, and optimize scalable data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data.
• Build and maintain data warehousing solutions and data lakes using big data technologies (e.g., Hadoop, Spark, Kafka).
• Implement data quality frameworks and ensure the accuracy, consistency, and reliability of data.
• Develop ETL/ELT processes and solutions to support business intelligence and analytics requirements.
• Collaborate with data scientists, analysts, and business stakeholders to understand data needs and deliver solutions.
• Monitor and troubleshoot data pipeline performance and data infrastructure issues.
• Optimize data storage and processing for cost-efficiency and performance.
• Stay abreast of emerging big data technologies and best practices, recommending and implementing new tools and techniques.
• Ensure data security and compliance with relevant regulations.

Requirements

• Bachelor's or Master's degree in Computer Science, Data Science, Engineering, or a related quantitative field.
• Minimum of 5 years of experience in data engineering, with a strong focus on big data technologies.
• Proficiency in programming languages such as Python, Scala, or Java.
• Hands-on experience with distributed data processing frameworks like Apache Spark, Hadoop ecosystem (HDFS, MapReduce, Hive).
• Experience with cloud data platforms (e.g., AWS EMR, S3, Redshift; Azure Data Lake, Databricks; GCP BigQuery).
• Strong SQL skills and experience with various database technologies (relational and NoSQL).
• Familiarity with data warehousing concepts and ETL/ELT tools.
• Understanding of data modeling techniques and best practices.
• Excellent problem-solving skills and ability to work independently.
• Strong communication skills for collaborating with technical and non-technical teams.

Benefits

• Competitive salary and comprehensive benefits package.
• Fully remote work arrangement with flexible hours.
• Allowance for home office setup.
• Opportunities for professional development, training, and conferences.
• Cutting-edge technology stack and challenging projects.
• Collaborative and innovative team culture.
• Generous paid time off.
• Retirement fund participation.