Principal Engineer, Data Engineering

WESTERN DIGITAL (SINGAPORE) PTE. LTD.; Singapore; On-site; SGD 90,000 - 130,000; Full time

Job summary

WESTERN DIGITAL (SINGAPORE) PTE. LTD. is seeking a data engineer to build and maintain ML-ready data pipelines from engineering, sensor and operational data. You will implement feature engineering, data quality frameworks, and robust data governance to ensure reliable model training data.

The role includes real-time sensor data ingestion, synthetic data workflow support, and close collaboration with the ML team to ensure end-to-end data integrity and reproducibility within the AI platform.

Qualifications

• 2–3 years of experience building end-to-end data pipelines.

Responsibilities

• Feature engineering pipelines to transform raw sensor and engineering data into ML-ready feature sets.
• Data quality checks covering completeness, schema consistency, and distribution stability.
• Data versioning and lineage to ensure reproducibility of model training.
• Define and govern data contracts between upstream producers and downstream ML consumers.
• Contribute to real-time streaming data ingestion and synthetic data pipeline infrastructure.
• Build and integrate training data registries, feature stores, and model inputs with the AI platform.

Job description

The data you will build pipelines for is not transactional data or clickstream data. It is experimental measurement data from precision product development instruments - each data point costs real time and resources to generate. Getting the data infrastructure right for this kind of scientific data is a genuinely different engineering challenge from standard web-scale or financial data work. You will develop rare expertise in ML-ready scientific data pipelines that very few data engineers in Singapore or globally have built.

Key Responsibilities

• Feature Engineering Pipelines: Build and maintain reliable, versioned feature engineering pipelines that transform raw engineering, sensor, and operational data into structured ML-ready feature sets - delivered to the specification defined
• Data Quality Frameworks: Design and operate data quality checks covering completeness, schema consistency, statistical distribution stability, and label accuracy across all AI training datasets. Alert the ML team when data quality degrades before it impacts model training. Collaborate with team who performs final downstream validation.
• Data Versioning, Lineage & Drift Detection: Build and maintain training data versioning and lineage tracking - ensuring full reproducibility of all model training runs and early alerting when deployment data diverges from training distributions.
• Data Contracts & Governance - Guided Implementation: Implement and maintain agreed data contracts between upstream data producers and downstream ML consumers, following governance standards established with guidance from ML Engineer. Establish access control and retention practices for all AI data assets.
• Real-Time Streaming - Sensor Data Ingestion: Contribute to real-time sensor data ingestion pipelines under technical direction. Develops operational ownership progressively over 6-12 months. Not a solo day-1 requirement.
• Synthetic Data Pipeline Support: Build pipeline infrastructure to operationalize synthetic data generation workstreams. With generative model methodology provided, builds ingestion, storage, and versioning infrastructure.
• MLOps Data Layer: Build and maintain the training dataset registry, feature store, and model input validation - tightly integrated with the AI platform (AWS Kubernetes, PortKey, Agent Gateway, LangFuse, AWS Guardrails, Elastic Search etc.).

Qualifications

Requirements:

Education

Bachelor's or Master's degree in AI, Computer Science, Data Engineering, Electrical Engineering, Applied Mathematics, or related field. AI major preferred; strong data engineering fundamentals required.

Experience

At least 2-3 years Demonstrated project experience building end-to-end data pipelines - academic, personal, or internship contexts - is the primary evaluation criterion. Python, SQL, and pipeline design fundamentals must be solid and demonstrable through project evidence.

Must Have Skills:

• Python: Strong proficiency - primary pipeline development language
• SQL: Strong proficiency - complex queries, window functions, data transformation logic
• Scalable Pipeline Design: Batch pipeline architecture; reliability, schema management, fault tolerance; Pipeline orchestration (Airflow, Prefect, AWS Glue Jobs)
• Data Quality Principles: Completeness checks, schema validation, distribution stability monitoring
• Data Versioning & Lineage: Reproducibility of training data; ability to trace data origin and transformations
• ML Data Lifecycle Awareness: Basic understanding of how data pipelines connect to ML model training. Awareness that data quality and pipeline design affect model performance downstream - specifically, awareness of risks like train/test data leakage and label quality impact on model accuracy. Does not require prior ML work experience; requires curiosity and conceptual understanding.

Good to have Skills:

• Feature store (Feast, Tecton) - Synthetic data generation (VAE, GAN, diffusion models) - Data Built Tool (DBT) - Data Load Tool DLT - Annotation platform integration (Label Studio, CVAT) - MES / LIMS system integration - Active learning data loop design - Data lakehouse (Iceberg, Dremio, AWS Glue, AWS Lake Formation) / Redhsift