Senior Data Engineer - Big Data Platforms

Placements24; Kimberley; On-site; CAD 120,000 - 160,000; Full time

Benefits offered by this job

Competitive salary

Medical aid and retirement savings

Generous annual leave and holidays

Professional development opportunities

Access to cutting-edge big data tech

Office in Kimberley

Job summary

Placements24 invites applications for a Senior Data Engineer to join our Kimberley headquarters. The role focuses on designing and managing scalable data pipelines, ensuring data quality and governance, and enabling analytics and BI initiatives across the organization.

You will collaborate with data scientists, analysts, and business stakeholders to shape our data strategy. Ideal candidates have 5+ years in data engineering with strong hands-on experience in distributed processing

Qualifications

• Master's degree in Computer Science, Engineering, or a related quantitative field.

Responsibilities

• Design, build, and maintain scalable data pipelines for batch and real-time data processing.
• Develop and optimize ETL/ELT processes using big data technologies.
• Implement and manage data warehousing solutions and data lakes.
• Ensure data quality, integrity, and security across all data platforms.
• Collaborate with data scientists and analysts to understand data requirements and provide solutions.
• Optimize database performance and query efficiency.
• Work with cloud platforms (AWS, Azure, GCP) for storage and processing.
• Implement data governance policies and procedures.
• Monitor data pipeline performance and troubleshoot issues promptly.
• Stay current with emerging trends in big data and data engineering.

Skills

Data engineering

Big data

SQL

Python

Scala/Java

ETL/ELT

Airflow

Cloud platforms

Data governance

Problem solving

Communication

Education

Master's degree

Tools

Apache Spark

Hadoop

Flink

Airflow

SQL Databases

NoSQL

AWS S3/Redshift/EMR

Azure Data Lake/Synapse

GCP BigQuery

Job description

About the Role

Our client, a major financial services institution based in Kimberley, is seeking a highly experienced Senior Data Engineer to build and manage their sophisticated big data infrastructure. This is a critical, office-based role within our Kimberley headquarters, requiring close collaboration with data scientists, analysts, and business stakeholders. You will be responsible for designing, developing, and optimizing data pipelines, ensuring data quality, scalability, and accessibility for analytics and business intelligence initiatives. The ideal candidate possesses extensive experience with distributed data processing systems, cloud data platforms, and a deep understanding of data warehousing principles. You will play a key role in shaping our data strategy and empowering the organization with data-driven insights. Strong technical acumen, problem-solving skills, and a commitment to data governance are essential for success in this role.

Key Responsibilities

• Design, build, and maintain scalable and reliable data pipelines for batch and real-time data processing.
• Develop and optimize ETL/ELT processes using big data technologies.
• Implement and manage data warehousing solutions and data lakes.
• Ensure data quality, integrity, and security across all data platforms.
• Collaborate with data scientists and analysts to understand their data requirements and provide solutions.
• Optimize database performance and query efficiency.
• Work with cloud platforms (e.g., AWS, Azure, GCP) for data storage and processing.
• Implement data governance policies and procedures.
• Monitor data pipeline performance and troubleshoot issues promptly.
• Stay current with emerging trends and technologies in big data and data engineering.

Requirements

• Master's degree in Computer Science, Engineering, or a related quantitative field.
• 5+ years of professional experience as a Data Engineer, with a strong focus on big data technologies.
• Proven experience with distributed data processing frameworks like Apache Spark, Hadoop, or Flink.
• Expertise in SQL and experience with various database technologies (relational and NoSQL).
• Hands-on experience with cloud data services (e.g., AWS S3, Redshift, EMR; Azure Data Lake, Synapse; GCP BigQuery).
• Proficiency in programming languages commonly used in data engineering (e.g., Python, Scala, Java).
• Experience with ETL/ELT tools and data pipeline orchestration (e.g., Airflow).
• Strong understanding of data modeling, data warehousing, and data lake concepts.
• Excellent problem-solving and analytical skills.
• Ability to work effectively in an office-based team environment in Kimberley.
• Strong communication and collaboration skills.

Benefits

• Highly competitive salary package and performance bonuses.
• Comprehensive medical aid and retirement savings plan.
• Generous annual leave and paid public holidays.
• Opportunities for professional development and advanced training.
• Access to cutting-edge big data technologies and infrastructure.
• Dynamic and collaborative work environment in our Kimberley office.
• Clear career progression paths within the organization.
• Company-sponsored social events and team activities.