Senior Data Engineer: Data Platform & Ops Leader (Hybrid)

bp; Kuala Lumpur; Ibrido; MYR 180.000 - 320.000; Tempo pieno

Vantaggi offerti da questo lavoro

Life and health insurance

Learning opportunities

Social communities

Descrizione del lavoro

bp is seeking a senior data/engineering leader in Malaysia to own and operate scalable data platforms and pipelines. You will guide architecture, drive DataOps practices, and mentor engineers while ensuring reliability, security, and governance across data products.

You will collaborate with data scientists, analysts, and stakeholders to deliver high-quality data solutions and maintain an end-to-end data lifecycle.

Competenze

• 8+ years of experience in Data Engineering, Software Engineering, or related disciplines.
• 5+ years of hands-on Python development experience.
• 3+ years of experience with PySpark and large-scale distributed data processing.
• Experience delivering and operating enterprise-scale data platforms and pipelines.
• Experience with DataOps, DevOps, SRE, or production support environments.
• Strong expertise in SQL, database design, and data modelling.
• Experience designing end-to-end data architectures including ingestion, transformation, storage, governance, and consumption layers.
• Strong AWS experience, including services such as S3, Glue, Lambda, Athena, Redshift, and EventBridge.
• Hands-on experience with Databricks and modern Lakehouse architectures.
• Experience with Git, CI/CD, Infrastructure as Code, and deployment automation.

Mansioni

• Drive engineering best practices through code reviews, technical standards, CI/CD practices, and automation.
• Lead source control, release management, and deployment processes for data engineering solutions.
• Facilitate technical discussions, architecture reviews, and technology decision-making across teams.
• Mentor engineers and contribute to building a high-performing engineering culture through knowledge sharing and technical leadership.
• Ensure compliance with information security, governance, and operational standard methodologies.
• Support the reliability, resilience, and continuous improvement of data platforms and services.
• Design, build, and operate scalable, secure, reliable, and efficient data products, pipelines, and platforms.
• Own the end-to-end lifecycle of data solutions, ensuring operational accountability, supportability, and continuous improvement.
• Collaborate with business stakeholders, engineers, architects, analysts, and data scientists to deliver high-quality data solutions.
• Design and maintain data models, ingestion frameworks, and transformation pipelines that ensure data quality, integrity, and performance.
• Implement DataOps best practices including CI/CD, Infrastructure as Code, automated testing, monitoring, and deployment automation.
• Build observability, alerting, and automated data quality controls to ensure platform reliability and rapid issue detection.
• Maintain metadata, lineage, documentation, runbooks, and support processes to improve governance and operational readiness.
• Lead incident resolution, root cause analysis, and reliability improvements to reduce recurring issues and improve service outcomes.
• Ensure compliance with security, privacy, governance, and enterprise architecture standards.
• Provide technical leadership through architecture guidance, mentoring, code reviews, and engineering best practices, including good team work

Conoscenze

Python

PySpark

SQL

Data Engineering

Data Modeling

AWS

Strumenti

Git

CI/CD

Terraform

Databricks

Lakehouse

Airflow

Descrizione del lavoro

bp is seeking a senior data/engineering leader in Malaysia to own and operate scalable data platforms and pipelines. You will guide architecture, drive DataOps practices, and mentor engineers while ensuring reliability, security, and governance across data products.

You will collaborate with data scientists, analysts, and stakeholders to deliver high-quality data solutions and maintain an end-to-end data lifecycle.