Lead Data Engineer: Lakehouse & AWS Expert

eTeam; Johor; Vor Ort; MYR 60.000 - 90.000; Vollzeit

Zusammenfassung

eTeam is seeking a data engineer to consolidate data from multiple enterprise systems, clean and reconcile it, and build robust data pipelines supporting OTIF calculations.

The role requires 5+ years in data engineering, with 2+ years in a technical lead position, strong Python/SQL, Spark/PySpark, and deep AWS experience across S3, Glue, Redshift, Athena, EMR, Lambda, Kinesis, and more. Extended 1-year contract with opportunity for renewal.

Qualifikationen

• 5+ years of data engineering experience with a leadership track record.
• Strong Python and SQL skills with hands-on coding.
• Extensive Spark/PySpark experience for large-scale processing.
• Deep AWS experience across multiple services and data platforms.
• Experience designing data lake/lakehouse architectures and ETL/ELT pipelines.
• Knowledge of CDC, batch and streaming architectures.
• Familiarity with relational and NoSQL databases and data warehousing concepts.

Aufgaben

• Consolidate data from multiple enterprise systems into reliable pipelines.
• Lead technical work and mentor teammates for data engineering tasks.
• Design and implement data lake/lakehouse solutions supporting OTIF calculations.
• Ensure data quality, lineage, and scalable architectures.

Kenntnisse

5+ years of data engineering

Technical lead

Python

SQL

Apache Spark

PySpark

Data modeling

Data warehouse concepts

ETL/ELT

CDC

Batch processing

Streaming architectures

Relational databases

NoSQL databases

Tools

AWS S3

AWS Glue

AWS Redshift

AWS Athena

AWS EMR

AWS Lambda

AWS Kinesis

AWS MSK

AWS Step Functions

AWS Lake Formation

Data Lake/Lakehouse

SQL databases

NoSQL databases

Jobbeschreibung

eTeam is seeking a data engineer to consolidate data from multiple enterprise systems, clean and reconcile it, and build robust data pipelines supporting OTIF calculations.

The role requires 5+ years in data engineering, with 2+ years in a technical lead position, strong Python/SQL, Spark/PySpark, and deep AWS experience across S3, Glue, Redshift, Athena, EMR, Lambda, Kinesis, and more. Extended 1-year contract with opportunity for renewal.