Senior Data Engineer: Spark, Databricks & AWS Pipelines

NOVACLOUD SYSTEMS PTE. LTD.; Singapore; On-site; SGD 110,000 - 170,000; Full time

Job summary

NOVACLOUD SYSTEMS PTE. LTD. is seeking a data engineer to design, develop and maintain scalable data pipelines for structured and semi-structured data. You will leverage Databricks, PySpark, and Apache Spark for large-volume processing and build robust ETL/ELT workflows across AWS.

Expertise in SQL and Python with a focus on performance, reliability, and data quality is required, along with experience in Delta Lake and Delta Live Tables where available.

Qualifications

• 5+ years of experience in data engineering, ETL development, data integration, or related field.
• Strong hands-on Databricks and data engineering workloads.
• Proficient in PySpark and Apache Spark for distributed processing.
• Experience building ETL/ELT pipelines and data solutions.
• Advanced SQL skills with complex queries, joins, and performance tuning.
• Hands-on experience with AWS data engineering services.
• Familiar with Delta Lake and Delta Live Tables preferred.

Responsibilities

• Design, develop, and maintain scalable data pipelines for structured and semi-structured data.
• Build ETL/ELT workflows using modern platforms and enterprise tools.
• Transform data with SQL and Python emphasizing performance and accuracy.
• Work with Delta Lake and Delta Live Tables for reliable pipelines.
• Optimize Spark jobs and queries to boost processing efficiency.
• Collaborate with cross-functional teams to meet data requirements.

Skills

Databricks

PySpark

Apache Spark

SQL

Python

AWS

Delta Lake

Delta Live Tables

IBM DataStage

Informatica PowerCenter

Oracle DB

DB2

Lambda

Redshift

IAM

CloudWatch

Glue

EC2

CI/CD

Git

Jenkins

Agile SDLC

Tools

Databricks

PySpark

Spark

SQL

Python

AWS Cloud

Delta Lake

DLT

DataStage

Informatica PowerCenter

Oracle

DB2

Lambda

Redshift

IAM

Glue

CloudWatch

EC2

GIT

Jenkins

Job description

NOVACLOUD SYSTEMS PTE. LTD. is seeking a data engineer to design, develop and maintain scalable data pipelines for structured and semi-structured data. You will leverage Databricks, PySpark, and Apache Spark for large-volume processing and build robust ETL/ELT workflows across AWS.

Expertise in SQL and Python with a focus on performance, reliability, and data quality is required, along with experience in Delta Lake and Delta Live Tables where available.