Senior Data Engineer: Lakehouse & Pipelines Lead

KRIS INFOTECH PTE. LTD.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

KRIS INFOTECH PTE. LTD. seeks a senior data engineer to design and run production-grade ETL/ELT pipelines using PySpark. You will architect and manage a lakehouse across raw, curated, and consumption layers, and deploy with Kubernetes and Docker in on-prem environments.

You will optimize SQL Server data models, enforce RBAC and governance, and lead CI/CD practices while mentoring junior engineers and collaborating with Data Scientists and Analysts.

Qualifications

• 8+ years IT experience with 5+ years in data engineering or data pipeline development.
• Expert-level SQL with SQL Server-like optimization experience.
• Advanced Python for data processing and production pipelines.
• Kubernetes to design, deploy, and manage containerized data pipelines on-premise.
• Strong data modeling for relational and non-relational concepts.
• Experience with flexible lakehouse architectures and metadata management (Iceberg).
• CI/CD and DevOps practices for automated testing and deployment.
• ETL/ELT orchestration with Airflow or similar tools.
• Hands-on with at least one NoSQL DB (MongoDB, Cassandra).
• Spark/PySpark for distributed processing and performance optimization.
• Data security and governance including RBAC and masking.
• Autonomy on complex projects with high code quality.
• Excellent problem-solving and cross-functional collaboration.

Responsibilities

• Design and implement production-grade ETL/ELT data pipelines using Python and PySpark.
• Manage lakehouse architecture across raw, curated, and consumption layers.
• Deploy and operate pipelines with Kubernetes and Docker in on-prem environments.
• Leverage SQL Server expertise for data models and query optimization.
• Establish CI/CD pipelines with automated tests and monitoring.
• Enforce security, governance, and RBAC across data layers.
• Mentor engineers, conduct code reviews, set best practices.
• Collaborate with Data Scientists and Analysts to deliver datasets.

Skills

SQL proficiency

Python programming

Data modeling

CI/CD/DevOps

NoSQL experience

Distributed processing

Problem solving

Communication

Education

Bachelor's degree in Computer Science/IT/Engineering

Tools

Apache Airflow

Spark/PySpark

Kubernetes

MongoDB

Iceberg tables

Job description

KRIS INFOTECH PTE. LTD. seeks a senior data engineer to design and run production-grade ETL/ELT pipelines using PySpark. You will architect and manage a lakehouse across raw, curated, and consumption layers, and deploy with Kubernetes and Docker in on-prem environments.

You will optimize SQL Server data models, enforce RBAC and governance, and lead CI/CD practices while mentoring junior engineers and collaborating with Data Scientists and Analysts.