Senior Data Engineer - Big Data Pipelines (Hybrid)

Iqvia; Durban; Hybrid; ZAR 900,000 - 1,500,000; Full time

Job summary

Iqvia in South Africa is seeking a Senior Data Engineer to lead the design and implementation of robust big data solutions. This hybrid role is based in East London, Eastern Cape, ZA, and will build scalable data pipelines to support analytics and ML initiatives.

You will partner with data scientists and analysts, implement Spark/Hadoop-based solutions, and drive data quality and governance across data platforms while mentoring junior engineers.

Qualifications

• Proven experience as a Data Engineer with a strong focus on big data technologies.
• Proficiency in programming languages such as Python, Scala, or Java.
• Experience with distributed computing frameworks (e.g., Apache Spark, Hadoop).
• Hands-on experience with cloud platforms (AWS, Azure, or GCP) and their data services.
• Strong understanding of SQL and NoSQL databases.
• Experience with data warehousing concepts and tools.
• Familiarity with ETL/ELT tools and data pipeline orchestration.
• Excellent analytical, problem-solving, and communication skills.
• Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.
• Ability to work effectively in a hybrid environment, balancing in-office collaboration in East London with remote flexibility.

Responsibilities

• Design, build, and maintain scalable and reliable big data pipelines and data warehouses.
• Develop and optimize ETL/ELT processes for large datasets.
• Implement data quality checks and ensure data integrity.
• Work with distributed computing frameworks like Spark and Hadoop.
• Collaborate with data scientists and analysts to understand data requirements.
• Ensure the performance and efficiency of data infrastructure.
• Develop and manage data models for various analytical purposes.
• Implement security and compliance measures for data management.
• Stay current with emerging big data technologies and best practices.
• Provide technical leadership and mentor junior data engineers.

Skills

Python

Scala

Java

Spark

Hadoop

AWS

Azure

GCP

SQL

NoSQL

Data Warehousing

ETL/ELT

Education

Bachelor's degree in CS/Engineering or related

Master's degree

Job description

Iqvia in South Africa is seeking a Senior Data Engineer to lead the design and implementation of robust big data solutions. This hybrid role is based in East London, Eastern Cape, ZA, and will build scalable data pipelines to support analytics and ML initiatives.

You will partner with data scientists and analysts, implement Spark/Hadoop-based solutions, and drive data quality and governance across data platforms while mentoring junior engineers.