Principal Data Engineer

StarHub; Petaling Jaya; On-site; MYR 180,000 - 260,000; Full time

Job summary

StarHub is seeking a senior data engineering leader to design, deliver, and operationalize complex data pipelines on a cloud-native platform. You will drive DataOps, production reliability, and trusted self-service analytics across multiple business domains.

You will mentor engineers, establish engineering standards, and partner with Data Architecture, BI, and stakeholders to translate requirements into scalable data products.

Qualifications

• 7–10+ years of experience in cloud-native data engineering.
• Hands-on architecture, delivery and production support on AWS and Snowflake.
• Lead design reviews, code reviews, and mentoring with no formal people management.

Responsibilities

• Technical Delivery & Solution Design: Lead ingestion, transformation, and data modeling solutions.
• Engineering Standards & Quality: Enforce SQL, Python, DAG design, CI/CD, testing, observability, and cost-aware patterns.
• Operational Excellence: Own production stability, incident triage, RCA, and preventative actions.
• Reusable Enablement: Build reusable components, templates, and runbooks to raise velocity.
• Data Quality & Trusted Data: Embed data quality controls, validation, and lineage checks.
• Collaboration & Enablement: Work with architects, BI, governance, and data owners to shape requirements.
• Technical Leadership by Influence: Act as senior technical escalation point and coach engineers.

Skills

Cloud-native data engineering

Python

SQL

Airflow

Snowflake

Airbyte

AWS

Data modeling

DataOps

Observability

Production stability

Self-service analytics

Mentoring

Education

Bachelor's degree in Computer Science or related field

Tools

Snowflake

Airflow

Airbyte

SageMaker

AWS services (S3, Lambda, KMS, Transfer Family, CloudWatch)

Streamlit

Job description

To provide technical leadership within StarHub’s Digital Experience Platform (DXP) Data organization by designing, delivering, and operationalizing complex data pipelines, curated datasets, and reusable engineering patterns on the cloud-native data platform. This role drives technical excellence across data ingestion, transformation, modeling, DataOps, and production reliability to enable trusted, scalable, and self-service analytics across business domains.

Accountabilities:

• Own technical delivery of complex, high-impact data engineering initiatives across ingestion, transformation, modeling, and operational stabilization.
• Serve as the senior technical leader within the Data Engineering function, setting implementation direction, reviewing design quality, and uplifting engineering standards across the team.
• Drive production reliability, observability, and root-cause elimination for critical pipelines and datasets.
• Develop reusable engineering patterns, frameworks, and automation to improve delivery speed, quality, and maintainability.
• Partner with Data Architecture, Platform Engineering, Data Quality Stewards, BI, and business stakeholders to translate requirements into trusted and scalable data products.
• Coach and mentor engineers through design reviews, code reviews, troubleshooting, and day-to-day technical guidance without direct people management responsibility.

Responsibilities:

• Technical Delivery & Solution Design: Lead design and implementation of complex ingestion, transformation, and curated data model solutions across Datapipe, Snowflake, and AWS, ensuring scalable, reusable, and cost-efficient patterns.
• Engineering Standards & Quality: Establish and enforce practical engineering standards across SQL, Python, DAG design, CI/CD, testing, observability, RBAC-aware implementation, and cost-aware design.
• Operational Excellence: Own production stability for critical pipelines and datasets, including incident triage, recovery leadership, RCA, and preventative improvement actions.
• Reusable Enablement: Build reusable components, templates, runbooks, and agentic delivery patterns to reduce duplicated effort, improve maintainability, and raise engineering velocity.
• Data Quality & Trusted Data: Embed automated data quality controls into pipelines and curated layers, including validation, anomaly detection, reconciliation, and schema drift checks.
• Collaboration & Enablement: Work with architects, stewards, platform engineers, BI teams, and business stakeholders to shape requirements into implementable data contracts and trusted datasets for self-service analytics.
• Technical Leadership by Influence: Act as the senior technical escalation point for difficult engineering and production issues, while coaching Senior Data Engineers and Data Engineers through design and implementation guidance.

Team Scope/ Stakeholders:

• Scope: Complex pipelines, curated datasets, reusable engineering patterns, and production reliability across the DXP Data Platform (C360, Datapipe ingestion solution based on Apache Airbyte & Airflow, Snowflake, SageMaker, Cloud native skills).
• Decision Rights: Technical design decisions within assigned initiatives, implementation patterns, code quality expectations, incident recovery actions, and recommendations on engineering prioritization and standards uplift.
• Stakeholders: Data Engineering, Platform Engineering, Architecture & Governance, BI, Data Science, Data Quality Stewards, Business Data Owners, Infrastructure, Cybersecurity/ISO, and Application domain teams.
• Resources: Individual contributor role operating as the senior-most hands-on engineer within the Data Engineering team, with responsibility to guide and uplift engineers across Singapore, Malaysia and India through technical leadership.
• 7–10+ years of experience in cloud-native data engineering, with strong hands-on architecture, delivery, and production support experience on AWS & Snowflake.
• Strong track record delivering complex data engineering initiatives independently, with the ability to operate across both build and run responsibilities.
• Experience partnering with BI and business teams to design modelled datasets and enable self-service analytics.
• Demonstrated technical leadership through design reviews, code reviews, mentoring, and troubleshooting guidance without formal team management responsibility.
• Deep hands-on technical expertise, including:
• Snowflake: schema design, Streams/Tasks, Stored Procedures, UDFs, RBAC-aware development, performance tuning, cost monitoring, Cortex AI, and Streamlit.
• Airflow or similar data orchestration tools: DAG design, orchestration, scheduling, dependency management, retry patterns, and observability.
• Python and SQL: pipeline scripting, transformation logic, data validation, and operational tooling.
• ELT/ETL frameworks: Airbyte, Fivetran, and custom connector understanding or development.
• AWS services: S3 (data lake structures and archival), Lambda, KMS, Transfer Family, CloudWatch, and SageMaker.
• Demonstrated success delivering medallion architecture (Bronze/Silver/Gold) and enabling self-service data use cases.
• Experience implementing automated data quality controls, remediation workflows, and data lineage-aware engineering practices across enterprise datasets.
• Familiarity with machine learning or AI integration using platforms like AWS SageMaker.
• Proven ability to troubleshoot complex data issues, lead root-cause analysis, and improve production stability through mechanisms rather than repeated manual intervention.
• Track record of raising team engineering quality through reusable patterns, operational discipline, and technical coaching.