Senior Python Developer

ENVISION DIGITAL TECHNOLOGIES SDN BHD; Kuala Lumpur; Hybrid; MYR 150,000 - 200,000; Full time

Job summary

A technology firm in Kuala Lumpur is seeking a Senior Python Developer to design and optimize data processing pipelines. The ideal candidate will have at least 2 years of experience in Python and Django, along with strong skills in Java Spring Boot. Responsibilities include developing ETL pipelines, integrating APIs for data collection, and ensuring high-volume dataset processing. This role offers a conducive work environment with perks like 13th month salary, medical coverage, and hybrid work arrangements.

Benefits

13th Month Salary (AWS)

Flexi Allowance

Insurance, Medical, Dental & Optical Coverage

Parking & Transport Support

Hybrid Work Arrangement

Office Near LRT

Modern Work Culture

Dynamic & Supportive Environment

Qualifications

• More than 2 years of professional experience in Python and Django development.
• Strong programming experience in Java Spring Boot for backend development.
• Proven experience with large-scale data ingestion and processing.

Responsibilities

• Design and develop ETL pipelines using Python and Scrapy.
• Integrate external APIs for data collection.
• Develop backend services using Django and Spring Boot.
• Handle and optimize large-volume data processing.
• Implement automated reporting pipelines.

Skills

Python

Django

Java Spring Boot

Data processing

ETL systems

Scrapy

SQL

RESTful API

Linux

Docker

Job description

You can now search for people on SEEK. Make your profile public so you can be found by more employers. Update profile visibility

Add expected salary to your profile for insights

We are looking for a Senior Python Developer to design, implement, and optimize large-scale data ingestion, processing, and reporting pipelines.

The ideal candidate will have strong experience in Python, Django, and Java Spring Boot, with a proven ability to handle high-volume datasets (up to billions of transactions) efficiently.

This role also requires strong communication skills to coordinate with internal teams and external partners for data collection, system integration, and continuous improvement of data workflows.

Key Responsibilities

• Design and develop ETL pipelines using Python and Scrapy for multi-source data ingestion, transformation, and validation at scale.
• Integrate external APIs for structured data collection, including authentication, pagination, rate limiting, and error handling.
• Develop backend services using Django (Python) and Spring Boot (Java) to process, store, and automate large datasets.
• Handle and optimize large-volume data processing, ensuring scalability, reliability, and high performance (e.g., handling 1 billion+ transaction records).
• Implement automated reporting pipelines to generate and distribute accurate operational and analytical reports.
• Verify and maintain data integrity, ensuring accuracy, completeness, and consistency across systems.
• Monitor and optimize pipeline performance, including throughput, latency, and health metrics, with automated alerts.
• Automate data verification and reconciliation workflows for anomaly detection and quality assurance.
• Collaborate with various parties — vendors, internal technical teams, and business stakeholders — to collect information, clarify requirements, and resolve data issues.
• Document workflows and technical processes for data validation, transformation, and reporting.

Requirements

• More than 2 years of professional experience in Python, Django development, focusing on data processing and ETL systems.
• Strong programming experience in Java Spring Boot for backend and API development.
• Hands-on experience with Scrapy or similar frameworks for large-scale data ingestion.
• Proven experience processing and optimizing very large datasets (hundreds of millions to billions of records).
• Proficient in RESTful API integration and advanced request handling (authentication, pagination, retries).
• Strong knowledge of SQL (MySQL, PostgreSQL, or ClickHouse) and data modeling.
• Experience generating data reports using Python (e.g., Pandas, Matplotlib) or BI/dashboard tools.
• Familiarity with Linux, Docker, and scheduling systems (Airflow, Cronicle, cron, etc).
• Excellent communication and collaboration skills to coordinate with multiple internal and external stakeholders.

Nice to Have

• Experience with Kafka, Redis, or other message queues for high-throughput pipelines.
• Familiarity with object storage systems (MinIO, S3) and analytical databases (ClickHouse, StarRocks).
• Experience with monitoring and observability (Prometheus, Grafana, Sentry).
• Exposure to reporting tools like Metabase or Apache Superset.
• Multilingual proficiency (English and Chinese) for regional coordination.

What Success Looks Like

• Stable, high-performance ETL and reporting pipelines capable of handling 1 billion+ records.
• Automated and transparent data reporting that supports fast decision-making.
• Efficient data validation workflows and clear documentation.
• Smooth communication and collaboration across internal and external teams.
• Continuous improvements in data reliability and operational efficiency.

Unlock job insights

Your application will include the following questions:

• What’s your expected monthly basic salary?
• How many years\' experience do you have as a Python Developer?
• Which of the following programming languages are you experienced in?
• How would you rate your Mandarin language skills?
• Which of the following languages are you fluent in?

Perks and benefits 🎁 13th Month Salary (AWS), Flexi Allowance 🩺 Insurance, Medical, Dental & Optical Coverage 🚗 Parking & Transport Support 🏠 Hybrid Work Arrangement 🚆 Office Near LRT 🤝 Modern Work Culture 🌟 Dynamic & Supportive Environment

Researching careers? Find all the information and tips you need on career advice.