Senior Python & Spark AI Developer — Remote/Hybrid

Recruit-It; South Africa; Hybrid; ZAR 700,000 - 1,100,000; Full time

Benefits offered by this job

Flexible working arrangements

Comprehensive benefits

Team culture

Performance incentives

Job summary

Recruit-It is seeking a Python/Spark/AI Developer to replat legacy T-SQL onto Spark/Delta Lake pipelines and build APIs and AI features around them. You will write type-safe, spec-first Python and deliver robust data pipelines with testing and CI gates.

The role involves building Spark/PySpark pipelines, REST APIs with FastAPI, and rigorous parity validation against the legacy system. Hybrid/remote setup offered.

Qualifications

• Proficient in Python 3.12 with type hints and static analysis.
• Experience building Spark data pipelines with PySpark and Delta Lake.
• Strong SQL skills and ability to translate T-SQL into Spark SQL.

Responsibilities

• Build Spark/PySpark pipelines on Delta Lake and replat legacy T-SQL to Spark SQL.
• Write type-hinted, tested Python with pytest and CI lint/type-check gates.
• Develop REST APIs (FastAPI) with auth, idempotency, and status semantics.
• Validate migrated pipelines against the legacy system and provide parity evidence.
• Document changes using ADRs and maintain spec-first workflow.

Skills

Python

PySpark

Delta Lake

Spark SQL

SQL

pytest

FastAPI

Docker

Git CI

Education

Bachelor's degree in CS/Engineering

Tools

Poetry

uvicorn

Job description

Recruit-It is seeking a Python/Spark/AI Developer to replat legacy T-SQL onto Spark/Delta Lake pipelines and build APIs and AI features around them. You will write type-safe, spec-first Python and deliver robust data pipelines with testing and CI gates.

The role involves building Spark/PySpark pipelines, REST APIs with FastAPI, and rigorous parity validation against the legacy system. Hybrid/remote setup offered.