Senior Data Engineer (Python Developer)

PERSOL SINGAPORE PTE. LTD.; Singapore; On-site; SGD 80,000 - 120,000; Full time

Job summary

A leading recruitment agency in Singapore is seeking a Senior Python Developer to enhance data pipelines and analytics in a fast-paced trading environment. The ideal candidate will have over 6 years of experience in Python, unit testing, and working within Agile teams. Responsibilities include developing efficient data workflows, ensuring code quality, and collaborating with trading and data teams to optimize solutions. This role offers a challenging yet rewarding opportunity to drive critical trading operations.

Qualifications

• 6+ years of Python development experience in data pipelines and automation.
• Strong expertise in unit and integration testing frameworks.
• Good understanding of CI/CD and DevOps practices.

Responsibilities

• Develop and maintain Python-based data pipelines for trading.
• Implement unit testing to ensure code quality.
• Collaborate with teams to define scalable engineering solutions.

Skills

Python development

Unit testing

Integration testing

DevOps practices

Agile methodologies

Analytical skills

Tools

AWS

Kubernetes

Apache Spark

Snowflake

Apache Airflow

GitHub

Job description

Senior AI Engineer -Senior Data Engineer-Python Developer

Industry: Trading / Investment / Financial Services

Role Overview

We are looking for an experienced Python Developer to design, build, and optimize scalable data pipelines and analytics workflows in a high-performance trading environment. The ideal candidate has strong engineering fundamentals, excellent testing discipline, and experience working with modern data engineering tools. This role involves close collaboration with trading, data, and platform teams to deliver reliable, automated, and efficient data solutions that support trading operations and decision-making.

Key Responsibilities

• Develop, optimize, and maintain Python-based data pipelines, ETL/ELT processes, and analytics workflows.
• Implement robust unit testing and integration testing to ensure code quality and maintainability.
• Collaborate with trading and data teams to understand requirements and translate them into scalable engineering solutions.
• Automate deployment, monitoring, and alerting for trading-related data workflows.
• Troubleshoot performance bottlenecks and enhance data processing efficiency.
• Work within Agile teams to deliver features and improvements in iterative cycles.
• Ensure compliance with engineering best practices and security standards.

Required Skills & Experience (Must Have)

• 6+ years of experience in Python development (data pipelines, automation, ETL/ELT, analytics workflows).
• Strong experience in unit and integration testing frameworks.
• Good understanding of DevOps practices, CI/CD, and automation.
• Experience working in Agile environments.
• Strong analytical and debugging skills.

Good-to-Have Skills

• Exposure to AWS cloud and Kubernetes (K8s).
• Familiarity with modern data platforms:
• Snowflake, Databricks, Apache Spark, Hive
• Table formats: Delta Lake, Apache Iceberg
• Vector databases (FAISS, Pinecone, Weaviate)
• Experience with orchestration tools:
• Apache Airflow, Dagster, Prefect, Temporal
• Knowledge of GitHub workflows and Datadog monitoring.

Ideal Candidate Attributes

• Strong problem-solving and performance optimization mindset.
• Excellent communication and collaboration skills.
• Ability to work independently in a fast-paced and data-driven trading environment.
• Passion for automation, clean code, and reliable engineering practices.

PERSOL Singapore Pte Ltd • RCB No. 200007268E

EA License No. 01C4394 • EA Registration No. R1113321 (Jacob Tijo)