lead / Senior data engineer: Python / AWS / Databricks

Capcon Asia; Kuala Lumpur; Hybrid; MYR 100,000 - 130,000; Full time

Job summary

A leading technology spin-off company in Kuala Lumpur seeks a Lead Data Engineer. In this role, you will enhance cloud-based data architecture, develop reliable data pipelines, and coach junior engineers within a dynamic, entrepreneurial team. If you have extensive experience with AWS, Databricks, and Python, we encourage you to apply for this exciting opportunity.

Qualifications

• 8+ years in data engineering, leadership experience necessary.
• Proven ability in designing and building data solutions.

Responsibilities

• Design and improve cloud-based architecture to support data needs.
• Develop reliable data pipelines and automation workflows.
• Lead discussions and mentor junior engineers.

Skills

Databricks

Python

SQL

AWS

Problem-solving

Communication

Job description

Why are they awesome?

• Data driven technology spin off from a leading aviation strategy, logistics and air cargo consultancy
• Building data-driven SaaS tools that help airlines make informed commercial decisions from; routes, maintenance, demand planning, operations and forecast models

Products / Projects

• 2 core SaaS products that are mature - actively used by customers and continuously evolving.
• 1 successful data product is already in place that provides market insights from the supply side. A new data product targeting the demand side is currently in its initial phase, with significant development and scaling planned for H2 2025 and into 2026

Current challenges

• Strategizing and planning efficient large-volume data extraction from various sources
• Optimizing and enhancing the existing data infrastructure, architecture and workflows
• Setting and driving technical standards across the data engineering function

Overall responsibilities

• Take ownership of designing and continuously improving the cloud-based data architecture (including AWS, EKS, and Databricks) to support scalability, performance, and security needs
• Develop and upkeep of reliable data pipelines, automation and CI/CD workflows
• Lead technical discussions - identify and dissect complex problems and effectively communicate solutions across tech and non-tech stakeholders
• Coach & mentor junior engineers by leading with examples

Team Org & Culture

• Reporting to the Head of Data, based in Netherlands (you will still work KL hours)
• Entrepreneurial and ambitious team from Malaysia and Europe, which include ex management consultants
• The engineering hub in KL (15 staff) supports software development, data engineering and analytics - engineers who are results-driven, proactive and ready to wear multiple hats

Must Have

• Databricks, Python, SQL and AWS - or any cloud platforms like GCP / Azure etc.
• Experience in building rather than only maintaining
• Excellent communication and problem-solving skills

Nice to have

• Strong numerical intuition and comfort working with data
• Experience in the cargo or logistics industry

Grade / Level

• Lead Data Engineer (8+ years and above, must have lead experience)
• Senior Data Engineer (5+ years and above)

Work Arrangement

• Hybrid with 3 days work from office
• Based in Bangsar South / KL Gateway (Close to MRT / LRT)