Senior Python Spark AI Developer - Delta Lake & APIs Remote

WatersEdge Solutions; Johannesburg; Hybrid; ZAR 700,000 - 1,100,000; Full time

Job summary

WatersEdge Solutions is recruiting an experienced Python/Spark/AI Developer to join a technically driven team modernising large-scale data platforms in a hybrid/remote environment. This role focuses on replatforming legacy T-SQL workloads into Spark and Delta Lake pipelines, while building APIs and AI capabilities.

You’ll work primarily in Python with a strong emphasis on tested, type-safe development. Ideal for an intermediate-to-senior developer who enjoys solving complex data engineering

Qualifications

• 4+ years of professional Python development experience.
• 2+ years of production experience building Spark or distributed data pipelines.
• Strong modern Python development skills with type hints and static analysis.
• Experience with Pydantic models and structured data validation.
• Experience with Delta Lake or lakehouse technologies.

Responsibilities

• Build production-grade Spark and PySpark pipelines on Delta Lake.
• Replatform legacy T-SQL workloads into Spark SQL.
• Translate data logic with accurate semantics and tests.
• Write type-hinted Python with static analysis.
• Develop and maintain automated pytest suites across tests.
• Build REST APIs using FastAPI or similar frameworks.
• Implement API validation, authentication and idempotency.
• Work in Docker-based environments and follow GitHub flow.
• Document changes and ADRs; contribute to AI/data workflows.

Skills

Python

PySpark

APIs

Docker

pytest

Type hints

GitHub flow

CI/CD

Education

Bachelor's degree in Computer Science, Engineering or related

Tools

Apache Spark

Delta Lake

FastAPI

Pydantic

CI/CD tooling

Job description

WatersEdge Solutions is recruiting an experienced Python/Spark/AI Developer to join a technically driven team modernising large-scale data platforms in a hybrid/remote environment. This role focuses on replatforming legacy T-SQL workloads into Spark and Delta Lake pipelines, while building APIs and AI capabilities.

You’ll work primarily in Python with a strong emphasis on tested, type-safe development. Ideal for an intermediate-to-senior developer who enjoys solving complex data engineering