Capability Lead – Data Engineering

Horizontal Talent; Petaling Jaya; On-site; MYR 350,000 - 550,000; Full time

Job summary

Horizontal Talent, a US-based recruitment firm, is assisting a leading multinational client to hire a Global Capability Lead – Data Engineering. This senior role defines global Data Engineering standards and drives architecture across the organisation.

You will build scalable cloud data platforms and pipelines with Azure, Azure Databricks, and Python, collaborating with stakeholders, architects, and engineers worldwide to enable analytics, AI, and digital transformation.

Qualifications

• Bachelor’s or Master’s degree in CS, Engineering, IT, or a related field.
• 10+ years software/data engineering experience with a focus on data engineering.
• Strong Python, Azure, and Databricks hands-on experience.
• Solid understanding of cloud data architectures, data pipelines, and performance optimization.
• Proven experience defining engineering standards and architectural patterns across teams.
• Demonstrated technical leadership and mentoring abilities.
• Excellent stakeholder management and communication skills.

Responsibilities

• Define global Data Engineering standards, architecture, frameworks, and best practices across multiple teams and business domains.
• Drive the development and continuous improvement of scalable cloud data platforms and pipelines using Microsoft Azure, Databricks, and Python.
• Establish best practices for data modelling, pipeline design, orchestration, performance optimisation, and data integration.
• Provide technical leadership, guide architectural decisions, mentor engineers, and strengthen engineering capabilities across teams.
• Collaborate with Data & Analytics, IT, platform engineering, architecture, AI teams, and business stakeholders to translate business requirements into scalable technical solutions.
• Improve engineering maturity, reliability, security, performance, and operational excellence across the global data platform.
• Enable advanced analytics, AI applications, and data-driven decision-making through reliable and scalable data engineering solutions.

Skills

Python

Azure

Azure Databricks

Data Engineering

Cloud Architecture

Leadership

Stakeholder management

Education

Bachelor's or Master's in CS/Engineering

Tools

Azure

Databricks

Python

Job description

About Horizontal Talent

Horizontal Talent is a US-based recruitment and staffing company specialising in IT, Technology, Digital, Engineering, and professional talent solutions. We partner with organisations across various industries to connect them with skilled professionals who support business growth, innovation, and digital transformation.

About Horizontal Talent

Horizontal Talent is a US-based recruitment and staffing company specialising in IT, Technology, Digital, Engineering, and professional talent solutions. We partner with organisations across various industries to connect them with skilled professionals who support business growth, innovation, and digital transformation. We are currently assisting our client, a leading multinational organisation, in recruiting an experienced Global Capability Lead – Data Engineering to strengthen its global Data & Analytics capabilities.

About The Role

This is a senior technical leadership position responsible for defining and driving global Data Engineering standards, architecture, and best practices across the organisation.

You will lead the development of scalable cloud-based data platforms and pipelines, working with Microsoft Azure, Azure Databricks, and Python. You will collaborate with global stakeholders, architects, platform teams, and data engineers to establish engineering excellence and build a trusted data foundation that supports analytics, AI, and digital transformation initiatives.

Key Responsibilities

• Define and implement global Data Engineering architecture, standards, frameworks, and best practices across multiple teams and business domains.
• Drive the development and continuous improvement of scalable cloud data platforms and pipelines using Microsoft Azure, Databricks, and Python.
• Establish best practices for data modelling, pipeline design, orchestration, performance optimisation, and data integration.
• Provide technical leadership, guide architectural decisions, mentor engineers, and strengthen engineering capabilities across teams.
• Collaborate with Data & Analytics, IT, platform engineering, architecture, AI teams, and business stakeholders to translate business requirements into scalable technical solutions.
• Improve engineering maturity, reliability, security, performance, and operational excellence across the global data platform.
• Enable advanced analytics, AI applications, and data-driven decision-making through reliable and scalable data engineering solutions.

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related discipline.
• At least 10 years of software engineering experience, with a strong focus on Data Engineering.
• Strong hands-on experience with Python, Microsoft Azure, Azure Databricks, and modern data engineering technologies.
• Solid understanding of cloud data architecture, data pipelines, data modelling, orchestration, and performance optimisation.
• Proven experience defining engineering standards, architectural patterns, and best practices across multiple teams.
• Demonstrated technical leadership, including mentoring engineers and driving engineering excellence.
• Strong stakeholder management, communication, and collaboration skills, with the ability to influence technical and business stakeholders.
• Ability to translate strategic direction into practical technical implementation while balancing scalability, quality, and performance.