Senior Data Engineer - AI-Driven Lakehouse & GenAI Pipelines

evolution recruitment solutions pte. ltd.; Singapore; On-site; SGD 180,000 - 240,000; Full time

Job summary

Evolution Recruitment Solutions Pte. Ltd. seeks an experienced Data Engineer to implement enterprise-grade Lakehouse platforms and data products in a large-scale environment.

You will design and operationalize batch, streaming, CDC, and API-based ingestion pipelines to support NLP/AI analytics and multimodal data, ensuring governance and quality across distributed teams. You will work with Spark/PySpark, SQL, Python/Scala, and modern data tools (Databricks, Snowflake, Kafka, Airflow, Kubernetes)

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Technology, or related field.
• 8-12 years of experience in Data Engineering, Big Data, Lakehouse, or large-scale data platforms.
• Hands-on with Databricks, Snowflake, Cloudera, Azure, AWS, GCP, Huawei, or Alibaba Cloud.
• Experience building scalable data ingestion, transformation, streaming, CDC, and data quality pipelines.
• Strong skills in Spark, PySpark, SQL, Python, and/or Scala.
• Experience with Iceberg, Delta Lake, and object storage platforms.
• Knowledge of data governance, metadata, lineage, and data modelling.
• Familiarity with ML platforms and frameworks such as Spark MLlib, scikit-learn, XGBoost.

Responsibilities

• Implement and operationalize enterprise-scale Lakehouse platforms and data products.
• Design, develop, test, and maintain scalable data ingestion pipelines (batch, streaming, CDC, API).
• Develop multimodal data ingestion including content extraction from files, images, video, and audio.
• Build data contracts, SLAs, data quality controls, and governance standards.
• Work with Iceberg, Hudi, Delta Lake and open table formats.
• Support RAG, vector search, GenAI, and agentic AI use cases.
• Performance tuning, troubleshooting, and root-cause analysis across data platforms.
• Develop APIs, dashboards, and BI data exposure.
• Create technical runbooks and deployment guides; ensure DevSecOps compliance.
• Collaborate with distributed teams across projects; drive reliability and automation.

Skills

Spark/PySpark/SQL

Python/Scala

Data pipeline design

NLP/AI analytics

Agile/DevOps mindset

Communication

Education

Bachelor's degree in CS/Engineering/IT

Tools

Databricks

Snowflake

Cloudera

Azure/AWS/GCP

Kafka/Flink/Spark Streaming

Airflow

Trino/Dremio/Hive/Impala

Kubernetes/OpenShift/Docker

Terraform/Jenkins/Git/CI-CD

MLflow

Job description

Evolution Recruitment Solutions Pte. Ltd. seeks an experienced Data Engineer to implement enterprise-grade Lakehouse platforms and data products in a large-scale environment.

You will design and operationalize batch, streaming, CDC, and API-based ingestion pipelines to support NLP/AI analytics and multimodal data, ensuring governance and quality across distributed teams. You will work with Spark/PySpark, SQL, Python/Scala, and modern data tools (Databricks, Snowflake, Kafka, Airflow, Kubernetes)