Senior Data Engineer — On-Prem Data Pipelines & Lakehouse

Crédit Agricole CIB; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Crédit Agricole Corporate and Investment Banking (Crédit Agricole CIB) in Singapore seeks a Senior Data Engineer to design, build, and maintain robust on-premise data pipelines and lakehouse architectures, enabling data-driven decision-making across the organization.

You will lead data infrastructure initiatives, mentor engineers, and collaborate with Data Scientists, Analysts, and stakeholders to deliver datasets aligned with operational needs in a high-performing engineering team.

Qualifications

• Bachelor's degree in Computer Science, IT, Engineering, or related field with demonstrated continuous learning ethos.
• 8+ years IT experience; 5+ years hands-on data engineering or data pipeline development.
• Expert-level SQL proficiency with SQL Server, including query optimization, indexing, and performance tuning.
• Advanced Python programming skills for data processing, automation, and production-grade pipeline development.
• Kubernetes expertise – design, deploy, and manage containerized data pipelines in on-premise environments.
• Strong data modeling expertise – relational and non-relational concepts.
• Proven experience with flexible lakehouse/data lake architecture, Iceberg tables and optimization.
• CI/CD and DevOps practices – IaC, Git, automated testing, deployment pipelines.
• ETL/ELT orchestration with Apache Airflow or similar tools.
• Hands-on experience with at least one NoSQL database (MongoDB, Cassandra).
• Hands-on experience with Apache Spark and PySpark for distributed processing.
• Data security and governance – RBAC, masking, compliance frameworks.
• Proven ability to work autonomously on complex projects.

Responsibilities

• Design and develop autonomous, production-grade ETL/ELT data pipelines using Python and PySpark.
• Implement flexible lakehouse architecture across raw, curated, and consumption layers.
• Deploy and manage data pipelines using Kubernetes and Docker for on-premise environments.
• Leverage SQL Server expertise to design data models, write and optimize queries.
• Establish robust CI/CD practices for data pipelines including testing and monitoring.
• Enforce data security, governance, and RBAC across all data layers with auditability.
• Mentor junior engineers and conduct code reviews to promote best practices.
• Collaborate with Data Scientists and Analysts to deliver datasets meeting needs.
• Provide L3 support for complex data challenges and evaluate new tools.

Skills

Python

PySpark

Kubernetes

SQL Server

Data modeling

CI/CD

ETL/ELT orchestration

NoSQL databases

Apache Spark

Data governance

Autonomous work

Collaboration

Education

Bachelor's degree in Computer Science, IT, Engineering, or related field

Tools

SQL Server

Kubernetes

Docker

Apache Airflow

Kafka

Iceberg

MongoDB

Apache Spark

PySpark

Job description

Crédit Agricole Corporate and Investment Banking (Crédit Agricole CIB) in Singapore seeks a Senior Data Engineer to design, build, and maintain robust on-premise data pipelines and lakehouse architectures, enabling data-driven decision-making across the organization.

You will lead data infrastructure initiatives, mentor engineers, and collaborate with Data Scientists, Analysts, and stakeholders to deliver datasets aligned with operational needs in a high-performing engineering team.