Senior Data Engineer

bp; Kuala Lumpur; Hybrid; MYR 240,000 - 360,000; Full time

Benefits offered by this job

Generous salary package with annual/aw

15% EPF contributions

Work-life balance & flexible working

Collaborative environment with diverse

Career development & progression

16 weeks parental leave

Job summary

bp in Kuala Lumpur invites seasoned data engineering leaders to shape our cloud-native data platforms. You will architect end-to-end data solutions, oversee ingestion, storage, and orchestration, and guide teams across data science and software engineering.

The role requires 8+ years in data/software engineering, strong Python and PySpark skills, AWS expertise, and a proven track record in enterprise-scale deployments.

Qualifications

• 8+ years of experience in Data/Software Engineering or related fields.
• 5+ years with Python for data engineering and automation.
• 3+ years with PySpark and distributed processing at scale.
• Strong database design, SQL & relational modelling skills.
• Experience building enterprise-scale data pipelines and platforms.
• Cloud-native data solutions on AWS (S3, Glue, EMR, Lambda, Redshift).

Responsibilities

• Lead data engineering initiatives and drive architectural decisions.
• Mentor engineers and foster high-performing teams.
• Oversee CI/CD pipelines, Git workflows, and release management.
• Ensure data governance, security, and compliance.
• Collaborate across teams to deliver scalable data platforms.
• Evaluate new data technologies and drive continuous improvement.

Skills

Data Engineering

Architecture Design

Mentoring

Cross-functional Leadership

Cloud Architecture

Security & Governance

Communication

Tools

Python

PySpark

Databricks

SQL

AWS

S3

Glue

EMR

Redshift

Airflow

Git

CI/CD

Terraform

Job description

Job Description

• eviews, technical standards, CI/CD practices, and automation.
• Lead source control, release management, and deployment processes for data engineering solutions.
• Facilitate technical discussions, architecture reviews, and technology decision-making across teams.
• Mentor engineers and contribute to building a high-performing engineering culture through knowledge sharing and technical leadership.
• Ensure compliance with information security, governance, and operational standard methodologies.
• Support the reliability, resilience, and continuous improvement of data platforms and services.

Requirements

• 8+ years of experience in Data Engineering, Software Engineering, or related technical subject areas.
• 5+ years of hands-on experience with Python development for data engineering and automation solutions.
• 3+ years of hands-on experience with PySpark and distributed data processing frameworks in enterprise-scale environments.
• Strong expertise in Database Design, SQL, and Relational Data Modelling.
• Shown experience designing, building, and supporting enterprise-scale data pipelines and data platforms.
• Strong experience in data engineering architecture and solution design, including data lakes, data warehouses, Lakehouse architectures, data integration patterns, and scalable data processing solutions.
• Proven track record to design end-to-end data solutions covering data ingestion, transformation, storage, orchestration, governance, and consumption layers.
• Strong proven understanding of AWS cloud services and experience building, deploying, and supporting cloud-native data solutions.
• Experience with AWS data engineering services such as S3, Glue, EMR, Lambda, Athena, Redshift, Step Functions, Event Bridge, or equivalent technologies.
• Hands-on experience with Databricks, including development, optimization, and operational support of large-scale data processing workloads.
• Strong understanding of Solution Architecture, System Design, and Engineering Best Practices.
• Experience with Infrastructure as Code, CI/CD pipelines, configuration management, and deployment automation.
• Experience with source control and code management tools such as Git.
• Knowledge of service operations, observability, resilience engineering, disaster recovery, and operational support models.
• Strong understanding of information security, data governance, and data privacy best practices.
• Excellent communication, collaborator management, and collaboration skills.
• Proven track record to provide technical leadership, mentor engineers, influence architectural decisions, and contribute to high-performing teams.

What We Offer

It’s crucial to us that the differences we see in the world around us are reflected in our workplace. Who you are is what counts, not where you’re from or how you live your life. At bp, we support our people to learn and grow in a diverse and challenging environment. We believe that our team is strengthened by diversity. We are committed to fostering an inclusive environment in which everyone is respected and treated fairly. There are many aspects of our employees’ lives that are meaningful, so we offer benefits to enable your work to fit with your life.

• Generous salary package including an annual bonus program and individual performance-based incentives.
• Additional EPF contributions totaling 15%.
• Excellent work-life balance & flexible working arrangements!
• Collaborative environment that celebrates achievements, diversity, and culture
• Ongoing career development and progression opportunities in a global organization
• 16 weeks paid parental leave (4 weeks partner leave)

Travel Requirement

No travel is expected with this role

Relocation Assistance

This role is not eligible for relocation

Remote Type

This position is a hybrid of office/remote working

Skills

Cloud Platforms, Cloud Platforms, Collaboration, Communication, Configuration management and release, Continuous deployment and release, Creating a high performing team, Database Design, Digital Project Management, Documentation and knowledge sharing, Emerging technology monitoring, Facilitation, Information Security, Mentoring, Metrics definition and instrumentation, NoSql data modelling, Problem Solving, Relational Data Modelling, Risk Management, Scripting, Secure development, Service operations and resiliency, Software Design and Development, Solution Architecture, Source control and code management {+ 5 more}

Legal Disclaimer

We are an equal opportunity employer. We do not discriminate on the basis of protected characteristics like race, religion, color, sex, national origin, sexual orientation, veteran status or disability status. Individuals with an accessibility need may request an adjustment/accommodation related to bp's recruiting process (e.g., accessing the job application, completing required assessments, participating in telephone screenings or interviews, etc.). If you would like to request an adjustment/accommodation related to the recruitment process, please contact us.

If you are selected for a position and depending on your role, your employment may be contingent upon adherence to local policy. This may include pre-placement drug screening, medical review of physical fitness for the role, and background checks.