Hybrid Data Engineer: Scalable Pipelines & Insights

Ten Group; Cape Town; Hybrid; ZAR 600,000 - 900,000; Full time

Benefits offered by this job

Flexible work arrangements

Hybrid work options

Paid time off

Sabbaticals

Medical aid

Job summary

Ten seeks a Data Engineer to build a scalable data foundation enabling analytics, reporting and product decisions. You will design end-to-end data infrastructure from ingestion to modelling to delivery, partnering with analysts, engineering and client services to translate needs into solutions.

Responsibilities include establishing data quality, developing scalable pipelines and APIs, and delivering reliable data feeds.

Qualifications

• Bachelor’s degree in Computer Science, Software Engineering, Computer Engineering, or a related field.
• 5+ years’ experience building data pipelines using Python – strong hands-on coding experience is essential.
• Proven experience working with AWS, specifically Lambda and ECS.
• Strong knowledge and hands-on experience using Airflow for ETL/ELT data pipelines.
• Familiarity with frameworks and tools such as Spark and Pandas.
• Ability to work with both batch and, ideally, streaming data processing.
• Experience building APIs and data delivery solutions, with a strong focus on data cleansing, integration and quality.
• Extensive hands-on experience with SQL, including building transformations, validating data and optimising analytical workloads.
• Strong communication skills, with the ability to translate complex technical concepts into clear, actionable insights for business stakeholders.

Responsibilities

• Establish and maintain data quality checks, validation processes, and governance standards.
• Ensure consistent documentation across data sources, pipelines, and models.
• Build scalable pipelines that ingest, normalise, and transform data from multiple internal systems into a unified, central data store.
• Develop clean, well-structured data models ready for analysis and reporting.
• Build APIs and data delivery mechanisms that provide fast, reliable access to analytics, insights, and reports.
• Partner with Data Analysts to ensure data is easily consumable for reporting and decision-making.
• Develop ingestion APIs and solutions for importing external or client data into databases.
• Own and maintain Data Feeds, ensuring timely fixes and high reliability.
• Improve data accessibility, query performance, and database efficiency.
• Work closely with Engineering to align on best practices and architectural improvements.
• Model and analyse the impact of potential product or engineering roadmap items to support prioritisation and decision-making.
• Design and operate modern, scalable data infrastructure in AWS (or an equivalent cloud platform).
• Apply software engineering and DevOps best practices including CI/CD, infrastructure-as-code, monitoring, and automation.

Skills

Python

AWS

Airflow

Spark

Pandas

SQL

APIs

Education

Bachelor’s degree in CS/SE

Tools

Snowflake

MS SQL Server

PostgreSQL

AWS Lambda

AWS ECS

Airflow

GitHub

Job description

Ten seeks a Data Engineer to build a scalable data foundation enabling analytics, reporting and product decisions. You will design end-to-end data infrastructure from ingestion to modelling to delivery, partnering with analysts, engineering and client services to translate needs into solutions.

Responsibilities include establishing data quality, developing scalable pipelines and APIs, and delivering reliable data feeds.