Senior Data Engineer

Sciente International Pte Ltd; Singapore; On-site; SGD 120,000 - 200,000; Full time

Job summary

Sciente International Pte Ltd is seeking a Senior Data Engineer to join our Data & Analytics team in Singapore. You will design, build, and maintain robust on-premise data pipelines, architect flexible lakehouse solutions, and lead data infrastructure initiatives to enable data-driven decision-making across the organization.

The role requires deep expertise in SQL Server, NoSQL, Spark/PySpark, Python, and containerized pipelines using Kubernetes.

Qualifications

• Bachelor's degree in CS/IT/Engineering or related field.
• At least 5 years' hands-on data engineering and pipeline development.
• Hands-on SQL Server expertise: querying, indexing, performance tuning.
• Experience with NoSQL databases (MongoDB or Cassandra).
• Proficient in Spark and PySpark for large-scale processing.
• Strong Python skills for data pipelines and automation.
• Containerized pipelines with Kubernetes in on-prem environments.
• Data modeling knowledge for relational and non-relational schemas.
• Experience with Airflow or similar orchestration tools.
• Knowledge of data security, RBAC, masking, and compliance.

Responsibilities

• Design and build autonomous, production-grade ETL/ELT pipelines using Python and PySpark.
• Develop flexible lakehouse architectures across raw, curated, and consumption layers.
• Deploy and manage pipelines with Kubernetes and Docker on-premise.
• Leverage SQL Server expertise to optimize data models and queries.
• Establish CI/CD practices for data pipelines with automated tests and versioning.
• Enforce governance and RBAC across data layers and ensure auditability.
• Mentor junior engineers and conduct code reviews.
• Collaborate with Data Scientists and Analysts to deliver datasets.

Skills

SQL Server

NoSQL databases

Apache Spark / PySpark

Python

Kubernetes

Data modeling (relational & non-rel)

Airflow

Data security & governance

Education

Bachelor's degree in Computer Science/IT/Engineering or related field

Tools

Docker

Microsoft SQL Server

Apache Airflow

NoSQL (MongoDB/Cassandra)

Job description

Job Summary

• We are seeking a Senior Data Engineer to join our Data & Analytics team. The ideal candidate will design, build, and maintain robust on-premise data pipelines, architect flexible lakehouse solutions, and lead data infrastructure initiatives to enable data-driven decision-making across the organization.

Mandatory Skill-set

• Bachelor's degree in Computer Science, IT, Engineering, or related field.
• Must have 5 years of hands-on experience in data engineering and data pipeline development;
• Hands onexperience in SQL, with in-depth expertise in Microsoft SQL Server, including query optimization, indexing strategies, performance tuning, and troubleshooting;
• Hands-on experience with at least one NoSQL database, such as MongoDB or Apache Cassandra;
• Practical experience in distributed data processing using Apache Spark and PySpark, including performance optimization and large-scale data processing;
• Strong Python programming skills for data processing, automation, and developing scalable, reliable, production-grade data pipelines;
• Expertise in designing, deploying, and managing containerized data pipelines using Kubernetes in on-premises environments;
• Strong understanding of relational and non-relational data modeling concepts, including schema design and optimization;
• Hands-on experience with Apache Airflow or similar orchestration tools for scheduling, monitoring, and managing batch and real-time data pipelines;
• Knowledge of data security and governance practices, including role-based access control (RBAC), data masking, and compliance with relevant regulatory and organizational standards;

Desired Skill-set

Good to have experience with on-premise data virtualization or logical data warehouse concepts.

Good to have real-time streaming technologies (Kafka, Apache Flink);

Responsibilities

• Design and develop autonomous, production-grade ETL/ELT data pipelines using Python and PySpark that ingest, transform, and deliver high-quality data while maintaining integrity and performance standards.
• Implement and manage flexible lakehouse architecture across raw, curated, and consumption layers, including data partitioning, cataloging, and metadata management;
• Deploy and manage data pipelines using Kubernetes and Docker to ensure scalability, reliability, and efficient resource utilization in on-premise environments;
• Leverage strong SQL Server expertise to design optimal data models, write complex queries, and perform query optimization across the data platform;
• Establish and maintain robust CI/CD practices for data pipeline deployment, including automated testing, version control, and continuous monitoring;
• Enforce security, governance, and role-based access controls across all data layers while ensuring compliance and auditability;
• Mentor junior engineers, conduct code reviews, and establish best practices across the team;
• Collaborate with Data Scientists, Business Analysts, and stakeholders to deliver datasets aligned with operational and analytical needs;
• Provide L3 support and expert consultation for complex data challenges; evaluate and recommend new tools and practices to improve agility and performance.

EA Licence No. 07C5639