Databricks Data Engineer: Build End-to-End Pipelines

Anchor Search Group Pte Ltd; Singapore; On-site; SGD 90,000 - 150,000; Full time

Job summary

The Anchor Search Group Pte Ltd is seeking a Data Engineer to design, build, and support reliable data ingestion and transformation pipelines on AWS and Databricks. You will work with architects, engineers, QA, and business teams to deliver production-ready data pipelines.

Responsibilities include batch and streaming data pipelines, Bronze/Silver/Gold data layers using PySpark and SQL, and implementing data quality, logging, and recovery while optimizing performance and cost.

Qualifications

• 3+ years of experience in data engineering or enterprise data-platform delivery.
• Hands-on experience with Databricks, Apache Spark, PySpark, and SQL.
• Experience with Delta Lake and Bronze/Silver/Gold data layers.
• Proven ability to build production data pipelines with logging and recovery.
• Experience with AWS services (S3, IAM, Glue, Lambda, Kinesis).
• Strong understanding of data modelling, quality, and performance optimization.

Responsibilities

• Design and build batch and/or streaming pipelines using Databricks and AWS.
• Develop data processing across Bronze/Silver/Gold layers with Spark and SQL.
• Ingest data from APIs, files, databases, and object storage.
• Create reusable data-engineering components and datasets.
• Implement data validation, logging, error handling, and recovery.
• Optimize Spark jobs, clusters, and storage for cost and performance.
• Support CI/CD, SIT/UAT, production deployment and handover.
• Collaborate with platform, application, BI, and data-governance teams.

Skills

Data engineering

Databricks

Apache Spark

PySpark

SQL

Data modelling

Data quality

Performance optimization

Tools

Databricks

Delta Lake

AWS S3

AWS IAM

AWS Glue

AWS Lambda

AWS Kinesis

Job description

The Anchor Search Group Pte Ltd is seeking a Data Engineer to design, build, and support reliable data ingestion and transformation pipelines on AWS and Databricks. You will work with architects, engineers, QA, and business teams to deliver production-ready data pipelines.

Responsibilities include batch and streaming data pipelines, Bronze/Silver/Gold data layers using PySpark and SQL, and implementing data quality, logging, and recovery while optimizing performance and cost.