Lead / Senior data engineer: Python / AWS / Databricks

Capcon Asia; Kuala Lumpur; Hybrid; MYR 180,000 - 300,000; Full time

Job summary

Capcon Asia, a data-driven spin-off from aviation strategy and logistics, is seeking a Senior Data Engineer to own cloud data architecture on AWS, EKS and Databricks, building scalable data pipelines.

Based in Kuala Lumpur, you will work in a hybrid setup (3 days in the Bangsar South / KL Gateway office) and report to the Head of Data in the Netherlands, coordinating with teams in Europe and Malaysia.

Qualifications

• Databricks, Python, SQL and AWS - or any cloud platforms like GCP / Azure etc.
• Experience in building rather than only maintaining
• Excellent communication and problem-solving skills

Responsibilities

• Design and improve cloud-based data architecture on AWS, EKS, and Databricks for scalability, performance and security.
• Develop and maintain reliable data pipelines, automation and CI/CD workflows.
• Lead technical discussions and communicate solutions to tech and non-tech stakeholders.
• Coach and mentor junior engineers by leading with examples

Skills

Databricks

Python

SQL

AWS

Job description

About the job lead / Senior data engineer: Python / AWS / Databricks

Why are they awesome?

• Data driven technology spin off from a leading aviation strategy, logistics and air cargo consultancy


• Building data-driven SaaS tools that help airlines make informed commercial decisions from; routes, maintenance, demand planning, operations and forecast models

Products / Projects

• 2 core SaaS products that are mature - actively used by customers and continuously evolving.


• 1 successful data product is already in place that provides market insights from the supply side. A new data product targeting the demand side is currently in its initial phase, with significant development and scaling planned for H2 2025 and into 2026


• Strategizing and planning efficient large-volume data extraction from various sources


• Optimizing and enhancing the existing data infrastructure, architecture and workflows


• Setting and driving technical standards across the data engineering function

Overall responsibilities

• Take ownership of designing and continuously improving the cloud-based data architecture (including AWS, EKS, and Databricks) to support scalability, performance, and security needs


• Develop and upkeep of reliable data pipelines, automation and CI/CD workflows


• Lead technical discussions - identify and dissect complex problems and effectively communicate solutions across tech and non-tech stakeholders
  
• Coach & mentor junior engineers by leading with examples

Team Org & Culture

• Reporting to the Head of Data, based in Netherlands (you will still work KL hours)


• Entrepreneurial and ambitious team from Malaysia and Europe, which include ex management consultants


• The engineering hub in KL (15 staff) supports software development, data engineering and analytics - engineers who are results-driven, proactive and ready to wear multiple hats

Must Have

• Databricks, Python, SQL and AWS - or any cloud platforms like GCP / Azure etc.


• Experience in building rather than only maintaining


• Excellent communication and problem-solving skills

Nice to have

• Strong numerical intuition and comfort working with data


• Experience in the cargo or logistics industry


• Lead Data Engineer (8+ years and above, must have lead experience)


• Senior Data Engineer (5+ years and above)

Work Arrangement

• Hybrid with 3 days work from office


• Based in Bangsar South / KL Gateway (Close to MRT / LRT)