Senior Data Engineer

BP PLC; Kuala Lumpur; Hybrid; MYR 220,000 - 320,000; Full time

Benefits offered by this job

Life and health insurance

Learning opportunities

Career development

Job summary

BP plc in Kuala Lumpur is seeking a Senior Data Engineer for a hybrid office/remote role. You will drive engineering best practices, lead deployment processes, and mentor engineers to build a high-performing data platform team.

The role requires 8+ years in data/software engineering, 5+ years of Python, and 3+ years PySpark with large-scale data processing. Strong AWS, Databricks, and DevOps practice experience are essential.

Qualifications

• 8+ years of experience in Data Engineering, Software Engineering, or related disciplines.
• 5+ years of hands-on Python development experience.
• 3+ years of experience with PySpark and large-scale distributed data processing.
• Experience delivering and operating enterprise-scale data platforms and pipelines.
• Experience with DataOps, DevOps, SRE, or production support environments.

Responsibilities

• Drive engineering best practices through code reviews, technical standards, CI/CD practices, and automation.
• Lead source control, release management, and deployment processes for data engineering solutions.
• Mentor engineers and contribute to building a high-performing engineering culture through knowledge sharing.

Skills

Python

PySpark

AWS

Databricks

SQL

DataOps

DevOps

CI/CD

Git

Tools

Databricks

Terraform

Docker

CI/CD tooling

Job description

## Senior Data EngineerApply: This position is a hybrid of office/remote working: Malaysia - Kuala Lumpur: Full time: Posted Today: End Date: October 30, 2026 (6 days left to apply): RQ117295**Entity:**Technology**Job Family Group:**IT&S Group**Job Description:****Let me tell you about the role**At the heart of bp's technological endeavors is the Technology organization. We are the driving force behind the technology that fuels bp's operations, from upstream energy production to downstream delivery to our esteemed customers.Our diverse teams cater to a broad spectrum of interests, encompassing everything from infrastructure and backend services to customer-centric web and native applications.With a keen emphasis on agility, we foster using native AWS and Azure services as our infrastructure foundation, including serverless architectures. Our ethos is not just about using the best tech for the task but also encouraging an environment where our software and platform engineers thrive, learn, and continuously evolve.**Key Responsibilities*** Drive engineering best practices through code reviews, technical standards, CI/CD practices, and automation* Lead source control, release management, and deployment processes for data engineering solutions.* Facilitate technical discussions, architecture reviews, and technology decision-making across teams.* Mentor engineers and contribute to building a high-performing engineering culture through knowledge sharing and technical leadership.* Ensure compliance with information security, governance, and operational standard methodologies.* Support the reliability, resilience, and continuous improvement of data platforms and services.* Design, build, and operate scalable, secure, reliable, and efficient data products, pipelines, and platforms.* Own the end-to-end lifecycle of data solutions, ensuring operational accountability, supportability, and continuous improvement.* Collaborate with business stakeholders, engineers, architects, analysts, and data scientists to deliver high-quality data solutions.* Design and maintain data models, ingestion frameworks, and transformation pipelines that ensure data quality, integrity, and performance.* Implement DataOps best practices including CI/CD, Infrastructure as Code, automated testing, monitoring, and deployment automation.* Build observability, alerting, and automated data quality controls to ensure platform reliability and rapid issue detection.* Maintain metadata, lineage, documentation, runbooks, and support processes to improve governance and operational readiness.* Lead incident resolution, root cause analysis, and reliability improvements to reduce recurring issues and improve service outcomes.* Ensure compliance with security, privacy, governance, and enterprise architecture standards.* Provide technical leadership through architecture guidance, mentoring, code reviews, and engineering best practices, including good team work**What you will need to be successful (experience and qualifications)*** 8+ years of experience in Data Engineering, Software Engineering, or related disciplines.* 5+ years of hands-on Python development experience.* 3+ years of experience with PySpark and large-scale distributed data processing.* Shown experience delivering and operating enterprise-scale data platforms and pipelines.* Experience working in DataOps, DevOps, SRE, or production support environments. **Technical Skills*** Strong expertise in SQL, database design, and data modelling.* Experience designing end-to-end data architectures including ingestion, transformation, storage, governance, and consumption layers.* Strong AWS experience, including services such as S3, Glue, Lambda, Athena, Redshift, and EventBridge.* Hands-on experience with Databricks and modern Lakehouse architectures.* Experience with Git, CI/CD, Infrastructure as Code, and deployment automation.**Operational Excellence*** Strong understanding of observability, monitoring, alerting, and platform reliability.* Experience implementing data quality controls, lineage, metadata, and governance practices.* Experience with incident management, root cause analysis, and continuous service improvement.* Knowledge of security, privacy, compliance, and operational risk management.**Leadership*** Excellent stakeholder management and communication skills.* Shown ability to influence technical decisions, mentor engineers, and drive engineering excellence.* Demonstrated ownership mindset with a focus on reliability, quality, and business outcomes.**About bp**At bp, we provide the following environment and benefits to you:* A company culture where we respect our diverse and unified teams, where we are proud of our achievements and where fun and the attitude of giving back to our environment are highly valued.* Possibility to join our social communities and networks* Learning opportunities and other development opportunities to craft your career path* Life and health insurance, medical care packageAnd many other benefits. We are an equal opportunity employer and value diversity at our company. We do not discriminate based on race, religion, colour, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, perform crucial job functions, and receive other benefits and privileges of employment.**Travel Requirement**No travel is expected with this role**Relocation Assistance:**This role is not eligible for relocation**Remote Type:**This position is a hybrid of office/remote working**Skills:**Cloud Platforms, Cloud Platforms, Collaboration, Communication, Configuration management and release, Continuous deployment and release, Creating a high performing team, Database Design, Digital Project Management, Documentation and knowledge sharing, Emerging technology monitoring, Facilitation, Information Security, Mentoring, Metrics definition and instrumentation, NoSql data modelling, Problem Solving, Relational Data Modeling, Risk Management, Scripting, Secure development, Service operations and resiliency, Software Design and Development, Solution Architecture, Source control and code management {+ 5 more}.**Legal Disclaimer:**We are an equal opportunity employer. We do not discriminate on the basis of protected characteristics like race, religion, color, sex, national origin, sexual orientation, veteran status or disability status. Individuals with an accessibility need may request an adjustment/accommodation related to bp’s recruiting process (e.g., accessing the job application, completing required assessments, participating in telephone screenings or interviews, etc.). If you would like to request an adjustment/accommodation related to the recruitment process, please contact us. If you are selected for a position and depending upon your role, your employment may be contingent upon adherence to local policy. This may include pre-placement drug screening, medical review of physical fitness for the role, and background checks.**About bp:****Technology**Join us in creating, growing, and delivering innovation at pace, enabling us to thrive while transitioning to a net zero world. All without compromising our operational risk management.Working with us, you can do this by:• deploying our integrated capability and standards in service of our net zero and safety ambitions • driving our digital transformation and pioneering new business models • collaborating to deliver competitive customer-focused energy solutions • originating, scaling and commercialising innovative ideas, and creating ground-breaking new businesses from them • protecting us by assuring management of our greatest physical and digital risksBecause together we are:• Originators, builders, guardians and disruptors • Engineers, technologists, scientists and entrepreneurs • Empathetic, curious, creative and inclusive