Senior Data Engineer: Aws Pipelines & Ai Governance

Jvr Software

South Africa

Hybride

ZAR 739 000 - 1 052 000

Plein temps

Il y a 2 jours
Soyez parmi les premiers à postuler

Une candidature complète en une minute

Un CV et une lettre de motivation personnalisés, prêts à envoyer.

Passez les filtres ATS

Avantages offerts par ce poste

Hybrid work arrangement
Professional development opportunities
Health and retirement benefits
Referral bonuses

Résumé du poste

Jvr Software in Sandton, Gauteng, seeks a Senior Data Engineer to design, build, and maintain scalable data pipelines and infrastructure for BI, ML, and analytics. You will ingest data from APIs, files and databases, implement data quality controls, and contribute to lakehouse architectures in a hybrid work model.

The role requires 4–7+ years in data engineering, strong Python/SQL, AWS experience, and familiarity with AI and LLM platforms. Collaboration with engineers and data teams is essential.

Qualifications

  • Bachelor's degree in Computer Science, Information Systems, Engineering or related field.
  • 4–7+ years in data engineering with production pipelines.
  • Strong AWS data architecture experience.
  • Advanced Python and SQL.
  • Data modelling and warehousing.
  • Ingestion from relational databases, cloud storage, APIs and files.
  • Git, CI/CD and automation.
  • Agile or SAFe delivery experience.
  • Experience using AI in engineering work.
  • Power BI or similar BI tool experience.
  • Ability to explain constraints to stakeholders and engineers.
  • Experience with LLM platforms (OpenAI, Anthropic, Gemini, Bedrock).
  • Experience building agentic workflows with Playwright, Puppeteer, Scrapy or Selenium.
  • Understanding of APIs, containers, CI/CD, testing, secure engineering.
  • Experience orchestrating data pipelines on AWS/GCP/Azure services.

Responsabilités

  • Design and build scalable batch and near‑real‑time data pipelines.
  • Ingest and process data from APIs, databases, files, event streams, and web sources.
  • Develop and maintain large‑scale scraping and data acquisition systems.
  • Clean, validate, transform, and model data for analytics and AI applications.
  • Build LLM‑powered workflows and agentic AI solutions with structured outputs and RAG.
  • Support modern data lake, warehouse, and lakehouse architectures.
  • Implement monitoring, observability, schema validation, and data quality controls.
  • Collaborate with engineering teams to deliver secure, scalable production systems.
  • Plan, estimate, break features into tasks, and meet product quality bar.
  • Design and build ingestion/transformation pipelines across systems and domains.
  • Shape solutions considering cost, security, performance and support.
  • Coach less experienced data engineers.
  • Provide analytics, BI and data science teams with clear data structures and lineage.
  • Review code and configuration; simplify where possible.
  • Use Git, automated testing, CI/CD and structured releases.
  • Document enough for others to support the work.
  • Use AI in the engineering workflow on approved tools only; protect data.

Connaissances

Data engineering
Python
SQL
Data modelling
Cloud architecture
CI/CD
Agile/SAFe
AI in engineering
BI concepts
Stakeholder communication

Formation

Bachelor's degree in Computer Science / IT / Engineering

Outils

AWS
Docker
Kubernetes
GitHub Actions
Azure DevOps
ArgoCD
Nexus
Playwright
Puppeteer
Scrapy
Selenium
OpenAI
Anthropic
Gemini
Bedrock
BigQuery
S3
Lambda
Glue
Azure Data Factory
Synapse

Description du poste

Are you passionate about data, coding and creating innovative technical solutions? Do you enjoy solving problems, exploring new technologies and seeing your work make a real impact?

About the Role

Our client is a dynamic data analytics firm seeking a skilled Senior Data Engineer to join their team in Sandton, Gauteng. The role offers a hybrid work model, blending office collaboration with remote flexibility. You will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure that enable efficient data processing and analysis. This is a critical role in ensuring the availability and integrity of data for business intelligence, machine learning, and reporting initiatives.

Key Responsibilities
  • Design and build scalable batch and near‑real‑time data pipelines.
  • Experience specifically with Claude Code.
  • Ingest and process data from APIs, databases, files, event streams, and web sources.
  • Develop and maintain large‑scale scraping and data acquisition systems.
  • Clean, validate, transform, and model data for analytics and AI applications.
  • Build LLM‑powered workflows and agentic AI solutions using structured outputs, retrieval‑augmented generation (RAG), API and database integrations.
  • Support modern data lake, warehouse, and lakehouse architectures.
  • Implement monitoring, observability, schema validation, and data quality controls.
  • Collaborate with engineering teams to deliver secure, scalable, and maintainable production systems.
  • Plan, estimate, break features into tasks, and hit the quality bar agreed with the product owner.
  • Design and build ingestion and transformation pipelines across systems and domains, including batch and incremental processes. Handle error, monitoring, alerting, validation and reconciliation.
  • Shape solutions that fit the target data platform. Consider cost, security, performance and support impact before building.
  • Coach less experienced data engineers.
  • Give analytics, BI and data science a clear view of structures, availability and how a pipeline actually behaves.
  • Review code and configuration. Simplify what is already there.
  • Use Git, automated testing, CI/CD and structured releases.
  • Document enough that someone else can support the work.
  • Use AI in the engineering workflow on approved tools only, validate output and avoid putting client or proprietary data into a public model.
  • Stay accountable for what goes into production.
  • Join incident response and root‑cause work when a pipeline fails. Build privacy, access control and regulatory requirements from the start.
Requirements and Experience
  • A bachelor’s degree in Computer Science, Information Systems, Engineering or a related field. A relevant certification helps.
  • 4–7+ years in data engineering, with production pipelines and platforms you can talk through.
  • Strong AWS data architecture experience.
  • Advanced Python and SQL.
  • Data modelling, analytics‑oriented schema design and warehousing.
  • Ingestion from relational databases, cloud storage, APIs and files.
  • Git, CI/CD and automation (GitHub Actions, Azure DevOps, Docker, Kubernetes, ArgoCD, Nexus).
  • Agile or SAFe delivery experience.
  • Experience using AI in engineering work and knowledge of how to check it.
  • Power BI or a similar BI tool is useful.
  • Ability to explain a constraint to a business stakeholder and a design choice to another engineer.
  • Experience with LLM platforms such as OpenAI, Anthropic, Gemini, Bedrock.
  • Experience building agentic workflows involving tool use, retrieval systems, memory, and scraping frameworks such as Playwright, Puppeteer, Scrapy or Selenium.
  • Strong understanding of APIs, containers, CI/CD, testing and secure engineering practices.
  • Experience with orchestrating data pipelines using cloud services (AWS S3, Lambda, Glue, Azure Data Factory, Synapse, GCP BigQuery).
Salary and Contract

Reference number: NG61354. Contract: remote role. Salary: up to R650 per hour, salary negotiable based on experience.

Benefits
  • Competitive salary and potential for performance bonuses.
  • Hybrid work arrangement for a flexible work-life balance.
  • Opportunities for professional development and acquiring new skills.
  • Work on challenging and impactful data projects.
  • Collaborative environment with experienced data professionals.
  • Exposure to modern big data technologies and cloud solutions.
  • Comprehensive health and retirement benefits.
  • We pay cash for successful referrals.
Qualifications
  • IT Degree / Diploma / Equivalent experience.
  • Strong AWS data architecture experience.
  • Advanced Python and SQL skills.
  • Data modelling, analytics‑oriented schema design and warehousing experience.
  • Ingestion from relational databases, cloud storage, APIs and files.
  • Git, CI/CD and automation experience.
  • Agile or SAFe delivery experience.
  • Experience with AI in engineering work.
  • Power BI or similar BI tool experience.
  • Excellent communication and stakeholder engagement abilities.
Ideal Candidate
  • Strong academic background.
  • Stable career history demonstrating commitment and consistent growth.
  • Passion for leveraging data to drive business decisions.
  • Ability to work independently while collaborating effectively with cross‑functional teams.
Obtenez votre examen gratuit et confidentiel de votre CV.

ou faites glisser et déposez votre fichier ici.

Similar jobs

Postes similaires à comparer

Senior Data Engineer — Azure Data Platform & Analytics
Senior Data Engineer — Azure Data Platform & Analytics

Cf Consulting • Gauteng

Sur place
ZAR 700 000 - 1 000 000
Competitive salary
On-site work environment
Medical, dental, and vision insurance
+2
Senior Data Engineer - Big Data & Ai Infrastructure
Senior Data Engineer - Big Data & Ai Infrastructure

Placements24 • Johannesburg

Hybride
ZAR 900 000 - 1 600 000
Health insurance
Retirement plans
Paid time off
+2
Data Engineer Jhb - Northern Suburbs
Data Engineer Jhb - Northern Suburbs

Psybergate • Johannesburg

Sur place
ZAR 1 000 000 - 1 500 000
Senior Data Engineer - Johannesburg - Up To R1.08Mil Per Annum
Senior Data Engineer - Johannesburg - Up To R1.08Mil Per Annum

e-Merge IT Recruitment • Johannesburg

Sur place
ZAR 900 000 - 1 080 000
Comprehensive health insurance
Retirement benefits
Professional development opportunities
+1
Senior Data Engineer - Cape Town
Senior Data Engineer - Cape Town

Moyo Africa Consulting • Afrique du Sud

Sur place
ZAR 900 000 - 1 500 000
Competitive salary
Health insurance
Retirement benefits
+3
Senior Data Engineer
Senior Data Engineer

Sabenza IT & Recruitment • Pretoria

Sur place
ZAR 800 000 - 1 200 000
Data Engineering Lead
Data Engineering Lead

Blue Pearl PTY • Johannesburg

Sur place
ZAR 1 200 000 - 1 900 000
Senior Data Engineer
Senior Data Engineer

Placements24 • Paarl

Hybride
ZAR 900 000 - 1 200 000
Competitive salary and benefits
Health and wellness benefits
Retirement savings plan
+4
Ai Data Engineer - Big Data Pipelines
Ai Data Engineer - Big Data Pipelines

Placements24 • Polokwane

Sur place
ZAR 600 000 - 950 000
Health insurance
Paid time off
Professional development
+1
Senior Data Engineer (Big Data & ETL)
Senior Data Engineer (Big Data & ETL)

Placements24 • Sandton

Hybride
ZAR 900 000 - 1 600 000
Competitive salary
Hybrid work arrangement
Professional development opportunities
+2