Data Engineer - Databricks / Lakehouse _ Contract

NTT SINGAPORE PTE. LTD.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

NTT SINGAPORE PTE. LTD. is seeking an experienced Data Engineer to design and operationalise enterprise-scale data platforms, Lakehouse solutions and data products.

The role requires strong hands-on Data Engineering, Databricks, Apache Spark, PySpark, Python and SQL experience, with exposure to modern Lakehouse architectures, cloud platforms and large-scale data ingestion and processing. You will build scalable pipelines for AI-driven analytics and GenAI use cases.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Technology or a related discipline.
• Minimum 6 years of relevant experience in Data Engineering, Big Data, Data Lake, Data Warehouse or Lakehouse implementations.
• Strong hands-on experience with Apache Spark, PySpark, Python and SQL.
• Experience with Databricks, Snowflake, Cloudera, Azure, AWS or GCP.
• Strong experience developing ETL/ELT, data ingestion, transformation and data processing pipelines.
• Experience with Delta Lake, Iceberg or Hudi.
• Experience with Kafka, Spark Streaming, Flink, Hive or Airflow.
• Knowledge of data modelling, metadata management, data lineage, governance and data quality.
• Experience with Kubernetes, OpenShift, Docker, Jenkins, Git and CI/CD.
• Strong troubleshooting and performance optimisation skills.

Responsibilities

• Design, develop and operationalise enterprise Data Lake, Lakehouse and data engineering platforms.
• Develop scalable batch, streaming, CDC and API-based data ingestion pipelines.
• Build and maintain data pipelines using Spark, PySpark, Python, SQL and related Big Data technologies.
• Develop reusable foundation and business data products, including data contracts, SLAs and data quality controls.
• Implement data ingestion, transformation, reconciliation and data quality frameworks.
• Work with Lakehouse technologies such as Delta Lake, Apache Iceberg and Apache Hudi.
• Develop solutions for structured, semi-structured and unstructured data.
• Work with streaming and distributed data technologies such as Kafka, Spark Streaming, Flink, Airflow, Hive, Trino or Dremio.
• Design data architectures supporting AI, NLP, RAG, vector search and GenAI/agentic applications.
• Support ingestion, curation, governance and consumption of unstructured data for AI analytics.
• Expose data through APIs, event streams, dashboards and other channels.
• Perform performance tuning, troubleshooting and production support.
• Implement automated deployment and engineering practices using Docker, Kubernetes/OpenShift and CI/CD pipelines.
• Collaborate with architecture, engineering, analytics and business teams across projects.
• Prepare technical documentation and runbooks.
• Ensure solutions comply with engineering standards and security.

Skills

Spark

PySpark

Python

SQL

Databricks

Kafka

Kubernetes

Airflow

Delta Lake

GenAI

Education

Bachelor's degree in Computer Science, Engineering, Information Technology or a related discipline

Tools

Databricks

Snowflake

Azure

AWS

GCP

Kubernetes

OpenShift

Docker

Jenkins

Job description

Data Engineer - Databricks / Lakehouse

Location: Alexandra Building
Contract Duration: 12 Months
Experience: Minimum 6 years of relevant experience

About the Role

We are looking for an experienced Data Engineer to design, develop and operationalise enterprise-scale data platforms, Lakehouse solutions and data products.

The successful candidate will have strong hands-on experience in Data Engineering, Databricks, Apache Spark, PySpark, Python and SQL , with exposure to modern Data Lake/Lakehouse architectures, cloud platforms and large-scale data ingestion and processing.

The role involves building scalable batch and streaming pipelines, developing reusable data products, implementing data quality frameworks, and supporting modern analytics, AI and GenAI use cases.

Key Responsibilities

• Design, develop and operationalise enterprise Data Lake, Lakehouse and data engineering platforms .
• Develop scalable batch, streaming, CDC and API-based data ingestion pipelines .
• Build and maintain data pipelines using Spark, PySpark, Python, SQL and related Big Data technologies .
• Develop reusable foundation and business data products , including appropriate data contracts, SLAs and data quality controls.
• Implement data ingestion, transformation, reconciliation and data quality frameworks.
• Work with modern Lakehouse and open table technologies such as Delta Lake, Apache Iceberg and Apache Hudi .
• Develop solutions for structured, semi-structured and unstructured data, including extraction and processing of content from different file formats.
• Work with streaming and distributed data technologies such as Kafka, Spark Streaming, Flink, Airflow, Hive, Trino or Dremio .
• Design data architectures supporting AI, NLP, RAG, vector search and GenAI/agentic applications .
• Support ingestion, curation, governance and consumption of unstructured data for AI-driven analytics.
• Expose data through APIs, event streams, dashboards and other enterprise consumption channels.
• Perform performance tuning, troubleshooting, production support and root cause analysis.
• Implement automated deployment and engineering practices using Docker, Kubernetes/OpenShift and CI/CD pipelines .
• Work closely with architecture, engineering, analytics and business teams across multiple projects.
• Prepare technical documentation, deployment guides and operational runbooks.
• Ensure solutions comply with engineering standards, security requirements, DevSecOps controls and software delivery practices.

Requirements

• Bachelor's degree in Computer Science, Engineering, Information Technology or a related discipline.
• Minimum 6 years of relevant experience in Data Engineering, Big Data, Data Lake, Data Warehouse or Lakehouse implementations.
• Strong hands-on experience with Apache Spark, PySpark, Python and SQL .
• Experience with one or more enterprise data platforms such as Databricks, Snowflake, Cloudera, Azure, AWS or GCP .
• Strong experience developing ETL/ELT, data ingestion, transformation and data processing pipelines .
• Experience working with Data Lake/Lakehouse architectures and technologies such as Delta Lake, Iceberg or Hudi .
• Experience with distributed and streaming technologies such as Kafka, Spark Streaming, Flink, Hive or Airflow .
• Good understanding of data modelling, metadata management, data lineage, governance and data quality .
• Experience with containerisation and DevOps technologies such as Kubernetes, OpenShift, Docker, Jenkins, Git and CI/CD .
• Strong troubleshooting, performance optimisation and root cause analysis skills.
• Experience working in Agile environments and delivering enterprise-scale technology solutions.
• Strong communication, collaboration and stakeholder management skills.

Good to Have

• Experience developing data products or enterprise data marketplace solutions .
• Experience supporting RAG, vector search, GenAI, NLP or AI/ML data pipelines .
• Experience processing multimodal or unstructured data such as documents, images, audio and video.
• Knowledge of Scala or Java for data engineering.
• Experience with MLflow, Spark MLlib, scikit-learn or XGBoost .
• Experience developing internal engineering tools using Python, shell scripting, Flask or React .
• Exposure to Teradata, Netezza, Greenplum or other MPP migration programmes .
• Relevant certifications such as Databricks Certified Data Engineer, Azure Data Engineer Associate, Google Professional Data Engineer, SnowPro or DAMA CDMP .