Python / Spark / AI Developer

Perssol; Cape Town; Hybrid; ZAR 700,000 - 1,200,000; Full time

Job summary

Perssol in Cape Town seeks a seasoned Python and Spark engineer to build pipelines on Delta Lake, replatform legacy T-SQL to Spark SQL, and deliver tested, type-safe Python with CI quality gates.

You will develop FastAPI REST APIs, validate migrated pipelines against legacy systems, and work in a remote/hybrid setup chosen by you.

Qualifications

• 4+ years Python development.
• 2+ years production Spark/PySpark.
• Strong Python 3.12, pytest, Pydantic & modern tooling.
• Spark SQL, DataFrames, performance tuning & distributed architecture.
• Docker, Git & CI/CD.

Responsibilities

• Build Python & PySpark pipelines on Delta Lake.
• Replatform legacy T-SQL to Spark SQL.
• Develop tested, type-safe Python with CI quality gates.
• Build FastAPI REST APIs with authentication & job tracking.
• Validate migrated pipelines against legacy systems.

Skills

Python

Spark

PySpark

CI/CD

Docker

Git

FastAPI

Delta Lake

Education

Bachelor’s Degree in CS/Engineering

Tools

Docker

Git

CI/CD

Job description

Build Python & PySpark pipelines on Delta Lake

Replatform legacy T-SQL to Spark SQL

Develop tested, type-safe Python with CI quality gates

Build FastAPI REST APIs with authentication & job tracking

Validate migrated pipelines against legacy systems

Location:

Cape Town, Remote/Hybrid or In office candidate can decide

Job Type:

Permanent

Date:

2 days ago

Requirements

4+ years Python development

2+ years production Spark/PySpark

Strong Python 3.12, pytest, Pydantic & modern tooling

Spark SQL, DataFrames, performance tuning & distributed architecture

Docker, Git & CI/CD

Able to work independently in a spec-driven environment

Nice to Have

Observability Data Migration & Parity Testing Databricks Certification

Qualifications:

Bachelor’s Degree in Computer Science, Engineering or equivalent experience

Spark/Databricks certification advantageous