Data Engineer (Databricks) - Scalable Cloud Pipelines

PERCEPT SOLUTIONS PTE. LTD.; Singapore; On-site; SGD 80,000 - 110,000; Full time

Job summary

Percept Solutions is seeking a Data Engineer to design, build, and support reliable data ingestion and transformation pipelines on AWS and Databricks in Singapore. The role will work with architects, engineers, QA, and business teams to deliver data pipelines, curated datasets, data-quality controls, and production-ready operational processes.

You will collaborate with platform, application, BI, and data-governance teams to implement robust data processing across Bronze, Silver, and Gold layers,

Qualifications

• 3+ years of experience in data engineering or enterprise data-platform delivery.
• Hands-on experience with Databricks, Apache Spark, PySpark and SQL.
• Experience with Delta Lake and layered Bronze/Silver/Gold data architecture.
• Experience building production data pipelines with proper logging, error handling, and recovery.
• Experience with AWS services such as S3, IAM, Glue, Lambda, Kinesis, or related services.
• Good understanding of data modelling, data quality, and performance optimisation.

Responsibilities

• Design and build batch and/or streaming data pipelines using Databricks and AWS services.
• Develop data processing across Bronze, Silver, and Gold layers using SQL, Python/PySpark, and Delta Lake.
• Implement ingestion from APIs, files, databases, object storage, or other enterprise data sources.
• Develop data transformations, dimensional or curated datasets, and reusable data-engineering components.
• Implement data validation, reconciliation, error handling, logging, rerun/recovery, and monitoring.
• Optimise Spark jobs, clusters, partitioning, and storage for performance and cost.
• Support CI/CD, environment promotion, SIT/UAT, production deployment, documentation, and operational handover.
• Collaborate with platform, application, BI, and data-governance teams.

Skills

Data modelling

Data quality

Performance optimisation

Tools

Databricks

Apache Spark

PySpark

SQL

Delta Lake

AWS (S3/IAM/Glue)

Job description

Percept Solutions is seeking a Data Engineer to design, build, and support reliable data ingestion and transformation pipelines on AWS and Databricks in Singapore. The role will work with architects, engineers, QA, and business teams to deliver data pipelines, curated datasets, data-quality controls, and production-ready operational processes.

You will collaborate with platform, application, BI, and data-governance teams to implement robust data processing across Bronze, Silver, and Gold layers,