Data Engineer

EPS COMPUTER SYSTEMS PTE LTD; Singapore; On-site; SGD 90,000 - 150,000; Full time

Job summary

EPS COMPUTER SYSTEMS PTE LTD is seeking a Data Engineer to design, build, and support reliable data ingestion and transformation pipelines on AWS and Databricks. You will collaborate with architects, engineers, QA, and business teams to deliver curated datasets, data-quality controls, and production-ready operational processes.

The role focuses on building batch and streaming pipelines with Databricks, SQL, Python/PySpark, and Delta Lake across Bronze/Silver/Gold layers, ensuring robust logging,

Qualifications

• 3+ years of experience in data engineering or enterprise data-platform delivery.
• Hands-on experience with Databricks, Apache Spark, PySpark, and SQL.
• Experience with Delta Lake and layered data architecture such as Bronze/Silver/Gold.
• Experience building production data pipelines with proper logging, error handling, and recovery.
• Experience with AWS services such as S3, IAM, Glue, Lambda, Kinesis, or related services.
• Good understanding of data modelling, data quality, and performance optimisation.

Responsibilities

• Design and build batch and/or streaming data pipelines using Databricks and AWS services.
• Develop data processing across Bronze, Silver, and Gold layers using SQL, Python/PySpark, and Delta Lake.
• Implement ingestion from APIs, files, databases, object storage, or other enterprise data sources.
• Develop data transformations, dimensional or curated datasets, and reusable data-engineering components.
• Implement data validation, reconciliation, error handling, logging, rerun/recovery, and monitoring.
• Optimise Spark jobs, clusters, partitioning, and storage for performance and cost.
• Support CI/CD, environment promotion, SIT/UAT, production deployment, documentation, and operational handover.
• Collaborate with platform, application, BI, and data-governance teams.

Skills

Data engineering

Databricks

Apache Spark

PySpark

SQL

Delta Lake

AWS

Data modelling

Data quality

Performance tuning

Tools

Databricks

Apache Spark

SQL

Delta Lake

AWS S3

AWS IAM

AWS Glue

AWS Lambda

AWS Kinesis

Job description

Job Description

We are looking for a Data Engineer to design, build, and support reliable data ingestion and transformation pipelines on AWS and Databricks. The role will work with architects, engineers, QA, and business teams to deliver data pipelines, curated datasets, data-quality controls, and production-ready operational processes.

Key Responsibilities

• Design and build batch and/or streaming data pipelines using Databricks and AWS services.
• Develop data processing across Bronze, Silver, and Gold layers using SQL, Python/PySpark, and Delta Lake.
• Implement ingestion from APIs, files, databases, object storage, or other enterprise data sources.
• Develop data transformations, dimensional or curated datasets, and reusable data-engineering components.
• Implement data validation, reconciliation, error handling, logging, rerun/recovery, and monitoring.
• Optimise Spark jobs, clusters, partitioning, and storage for performance and cost.
• Support CI/CD, environment promotion, SIT/UAT, production deployment, documentation, and operational handover.
• Collaborate with platform, application, BI, and data-governance teams.

Qualifications

Requirements (Must-Have)

• 3+ years of experience in data engineering or enterprise data-platform delivery.
• Hands-on experience with Databricks, Apache Spark, PySpark, and SQL.
• Experience with Delta Lake and layered data architecture such as Bronze/Silver/Gold.
• Experience building production data pipelines with proper logging, error handling, and recovery.
• Experience with AWS services such as S3, IAM, Glue, Lambda, Kinesis, or related services.
• Good understanding of data modelling, data quality, and performance optimisation.

Plus Points (Advantage)

• Databricks Workflows, Delta Live Tables, Unity Catalog, Auto Loader, or Structured Streaming.
• Experience with APIs, XML/JSON, event or near-real-time ingestion.
• CI/CD and infrastructure/configuration automation using Git-based tools.
• Experience with Tableau, BI datasets, or data warehouse integration.
• Databricks, AWS, data engineering, AI/ML, or GenAI-related certification(s).

What We Value

• Strong engineering fundamentals and willingness to learn.
• Practical problem-solving and attention to data correctness.
• Clean implementation, documentation discipline, and reliable delivery.
• Ability to work in a fast-paced project environment and collaborate with different technical teams.
• Ownership of assigned tasks and openness to technical guidance and feedback.
• A good overall match is more important than meeting every listed technology requirement.