Senior Data Engineer

KRIS INFOTECH PTE. LTD.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

KRIS INFOTECH PTE. LTD. seeks a senior data engineer to design and run production-grade ETL/ELT pipelines using PySpark. You will architect and manage a lakehouse across raw, curated, and consumption layers, and deploy with Kubernetes and Docker in on-prem environments.

You will optimize SQL Server data models, enforce RBAC and governance, and lead CI/CD practices while mentoring junior engineers and collaborating with Data Scientists and Analysts.

Qualifications

• 8+ years IT experience with 5+ years in data engineering or data pipeline development.
• Expert-level SQL with SQL Server-like optimization experience.
• Advanced Python for data processing and production pipelines.
• Kubernetes to design, deploy, and manage containerized data pipelines on-premise.
• Strong data modeling for relational and non-relational concepts.
• Experience with flexible lakehouse architectures and metadata management (Iceberg).
• CI/CD and DevOps practices for automated testing and deployment.
• ETL/ELT orchestration with Airflow or similar tools.
• Hands-on with at least one NoSQL DB (MongoDB, Cassandra).
• Spark/PySpark for distributed processing and performance optimization.
• Data security and governance including RBAC and masking.
• Autonomy on complex projects with high code quality.
• Excellent problem-solving and cross-functional collaboration.

Responsibilities

• Design and implement production-grade ETL/ELT data pipelines using Python and PySpark.
• Manage lakehouse architecture across raw, curated, and consumption layers.
• Deploy and operate pipelines with Kubernetes and Docker in on-prem environments.
• Leverage SQL Server expertise for data models and query optimization.
• Establish CI/CD pipelines with automated tests and monitoring.
• Enforce security, governance, and RBAC across data layers.
• Mentor engineers, conduct code reviews, set best practices.
• Collaborate with Data Scientists and Analysts to deliver datasets.

Skills

SQL proficiency

Python programming

Data modeling

CI/CD/DevOps

NoSQL experience

Distributed processing

Problem solving

Communication

Education

Bachelor's degree in Computer Science/IT/Engineering

Tools

Apache Airflow

Spark/PySpark

Kubernetes

MongoDB

Iceberg tables

Job description

Design and develop autonomous, production-grade ETL/ELT data pipelines using Python and PySpark that ingest, transform, and deliver high-quality data while maintaining integrity and performance standards.

Implement and manage flexible lakehouse architecture across raw, curated, and consumption layers, including data partitioning, cataloging, and metadata management.

Deploy and manage data pipelines using Kubernetes and Docker to ensure scalability, reliability, and efficient resource utilization in on-premise environments.

Leverage strong SQL Server expertise to design optimal data models, write complex queries, and perform query optimization across the data platform.

Establish and maintain robust CI/CD practices for data pipeline deployment, including automated testing, version control, and continuous monitoring.

Enforce security, governance, and role-based access controls across all data layers while ensuring compliance and auditability.

Mentor junior engineers, conduct code reviews, and establish best practices across the team.

Collaborate with Data Scientists, Business Analysts, and stakeholders to deliver datasets aligned with operational and analytical needs.

Provide L3 support and expert consultation for complex data challenges; evaluate and recommend new tools and practices to improve agility and performance.

Requirements

Must-have qualifications:

• 8+ years IT experience; 5+ years hands-on data engineering or data pipeline development.
• Expert-level SQL proficiency with strong expertise in SQL Server, including query optimization, indexing, and performance tuning.
• Advanced Python programming skills for data processing, automation, and production-grade pipeline development.
• Kubernetes expertise - Design, deploy, and manage containerized data pipelines in on-premise environments.
• Strong data modeling expertise - Both relational and non-relational concepts.
• Proven experience with flexible lakehouse/data lake architecture - Multi-layer data lakes, partitioning strategies, and metadata management, Iceberg tables and optimization.
• CI/CD and DevOps practices - Setting up CI/CD pipelines, Git, automated testing, and infrastructure-as-code tools.
• ETL/ELT orchestration experience - Apache Airflow or similar tools for scheduling and monitoring batch and real-time jobs.
• Hands-on experience with at least one NoSQL database (MongoDB, Cassandra, etc.).
• Hands-on experience with Apache Spark and PySpark for distributed data processing and performance optimization.
• Data security and governance - Role-based access control, data masking, and compliance sframeworks.
• Proven ability to work autonomously on complex projects while maintaining high code quality standards.
• Excellent problem-solving, communication, and cross-functional collaboration skills.
• Bachelor's degree in Computer Science, IT, Engineering, or related field with demonstrated continuous learning ethos.

Preferred qualifications:

• Experience with on-premise data virtualization or logical data warehouse concepts.
• Understanding of data mesh or data fabric architecture patterns.
• Real-time streaming technologies (Kafka, Apache Flink).
• Metadata management and data lineage tools.
• Experience mentoring junior engineers or leading technical initiatives.
• Agile delivery methodologies and product-oriented data architecture.

Other Professional Skills and Mind-set:

• Autonomous Work Ethic - Work independently on complex problems while proactively seeking collaboration.
• Continuous Learning - Committed to staying current with data engineering trends and best practices.