Senior Data Engineer (Banking, 1-year renewable contract)

evolution recruitment solutions pte. ltd.; Singapore; On-site; SGD 180,000 - 240,000; Full time

Job summary

Evolution Recruitment Solutions Pte. Ltd. seeks an experienced Data Engineer to implement enterprise-grade Lakehouse platforms and data products in a large-scale environment.

You will design and operationalize batch, streaming, CDC, and API-based ingestion pipelines to support NLP/AI analytics and multimodal data, ensuring governance and quality across distributed teams. You will work with Spark/PySpark, SQL, Python/Scala, and modern data tools (Databricks, Snowflake, Kafka, Airflow, Kubernetes)

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Technology, or related field.
• 8-12 years of experience in Data Engineering, Big Data, Lakehouse, or large-scale data platforms.
• Hands-on with Databricks, Snowflake, Cloudera, Azure, AWS, GCP, Huawei, or Alibaba Cloud.
• Experience building scalable data ingestion, transformation, streaming, CDC, and data quality pipelines.
• Strong skills in Spark, PySpark, SQL, Python, and/or Scala.
• Experience with Iceberg, Delta Lake, and object storage platforms.
• Knowledge of data governance, metadata, lineage, and data modelling.
• Familiarity with ML platforms and frameworks such as Spark MLlib, scikit-learn, XGBoost.

Responsibilities

• Implement and operationalize enterprise-scale Lakehouse platforms and data products.
• Design, develop, test, and maintain scalable data ingestion pipelines (batch, streaming, CDC, API).
• Develop multimodal data ingestion including content extraction from files, images, video, and audio.
• Build data contracts, SLAs, data quality controls, and governance standards.
• Work with Iceberg, Hudi, Delta Lake and open table formats.
• Support RAG, vector search, GenAI, and agentic AI use cases.
• Performance tuning, troubleshooting, and root-cause analysis across data platforms.
• Develop APIs, dashboards, and BI data exposure.
• Create technical runbooks and deployment guides; ensure DevSecOps compliance.
• Collaborate with distributed teams across projects; drive reliability and automation.

Skills

Spark/PySpark/SQL

Python/Scala

Data pipeline design

NLP/AI analytics

Agile/DevOps mindset

Communication

Education

Bachelor's degree in CS/Engineering/IT

Tools

Databricks

Snowflake

Cloudera

Azure/AWS/GCP

Kafka/Flink/Spark Streaming

Airflow

Trino/Dremio/Hive/Impala

Kubernetes/OpenShift/Docker

Terraform/Jenkins/Git/CI-CD

MLflow

Job description

Please note that visa sponsorship is not available at this time.

Key Responsibilities

• Implement and operationalize enterprise-scale Lakehouse platforms, data products, and data marketplace capabilities.
• Design, develop, test, and maintain scalable batch, streaming, CDC, and API-based data ingestion pipelines.
• Develop multimodal data ingestion pipelines, including:
  • Content extraction from various file formats.
  • Regex-based extraction of specific fields.
  • Content extraction from embedded images.
  • Frame extraction from video files.
  • Tran extraction from audio files.
• Build and maintain foundation and business data products with defined data contracts, SLAs, data quality controls, and governance standards.
• Implement and work with modern open table formats, including Iceberg, Hudi, and Delta Lake.
• Develop and support data pipelines for RAG, vector search, Generative AI (GenAI), and agentic AI use cases.
• Perform performance tuning, optimization, production support, troubleshooting, and root cause analysis across data platforms and pipelines.
• Implement data ingestion, transformation, reconciliation, and data quality frameworks.
• Develop data architectures supporting NLP and AI-driven analytics, including the ingestion, curation, governance, and management of structured and unstructured data.
• Support ML platforms and workflows, including model development, deployment, and operationalization.
• Develop internal engineering tools and full-stack applications using Python, shell ing, and modern web frameworks.
• Expose and integrate data through APIs, event streams, dashboards, and BI platforms.
• Create and maintain technical documentation, deployment guides, and operational runbooks.
• Ensure compliance with engineering standards, DevSecOps controls, CI/CD practices, security requirements, and software delivery standards.
• Collaborate effectively with distributed engineering, data, technology, and business teams across multiple projects.
• Drive continuous improvement in data platform reliability, scalability, automation, and operational excellence.

Key Requirements

• Bachelor's degree in Computer Science, Engineering, Information Technology, or a related discipline.
• 8-12 years of experience in Data Engineering, Big Data, Data Lake, Lakehouse, or large-scale data platform implementations.
• Strong hands-on experience with enterprise data platforms such as Databricks, Snowflake, Cloudera, Azure, AWS, GCP, Huawei, or Alibaba Cloud.
• Hands-on experience designing and developing data products and data marketplace capabilities.
• Strong expertise in Spark, PySpark, SQL, Python, and/or Scala, with strong overall programming skills.
• Proven experience building scalable data ingestion, transformation, streaming, CDC, reconciliation, and data quality pipelines.
• Hands-on experience with Iceberg, Hudi, Delta Lake, and object storage platforms.
• Experience with modern data technologies such as Kafka, Flink, Spark Streaming, Airflow, Trino, Dremio, Hive, and/or Impala.
• Strong hands-on experience with Kubernetes, OpenShift, Docker, Terraform, Jenkins, Git, CI/CD, MLflow, and observability tools.
• Experience designing data architectures and pipelines supporting NLP, AI-driven analytics, RAG, vector search, GenAI, and agentic AI use cases.
• Experience handling and governing unstructured and multimodal data, including documents, images, audio, and video.
• Experience with ML platforms and frameworks such as CML, Spark MLlib, scikit-learn, and XGBoost, including model deployment.
• Strong knowledge of data modelling, metadata management, data lineage, data governance, and data quality.
• Experience exposing data through APIs, event streams, dashboards, and BI platforms.
• Experience with full-stack/internal engineering tool development using Python, shell ing, Flask, React, or similar technologies is advantageous.
• Knowledge or experience with Teradata, Netezza, Greenplum, or MPP migration programmes is advantageous.
• Strong engineering, automation, troubleshooting, and performance optimization mindset.
• Strong communication and stakeholder management skills, with the ability to work effectively across distributed teams and multiple projects.
• Experience working in Agile delivery environments and enterprise-scale technology platforms.
• Strong commitment to quality, operational excellence, automation, and continuous improvement.
• Relevant certifications such as Databricks Certified Data Engineer, Azure Data Engineer Associate, AWS Data Analytics Specialty, Google Professional Data Engineer, SnowPro, or DAMA CDMP would be an advantage.