Senior Data Engineer, Databricks & AI Automation

Flintex Consulting Pte Ltd

Singapore

On-site

SGD 120,000 - 180,000

Full time

2 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Flintex Consulting Pte Ltd in Singapore seeks a Senior Data Engineer focused on Databricks and automation to design, build and operate reliable data pipelines for analytics, business applications and AI use cases. The role is hands-on with Databricks, Python, PySpark and SQL, and aims to simplify recurring engineering tasks through automation, using AI where it adds clear value.

You will collaborate with the Data and AI Innovation Team to deliver curated datasets, enforce data quality, monitor

Qualifications

  • 5+ years hands-on data engineering experience.
  • Python, SQL and Spark / PySpark proficiency.
  • Databricks and Delta Lake experience; understand job orchestration.
  • Data modelling, incremental ingestion and data quality.
  • Cloud data platform experience; Git/testing/deployment pipelines.
  • Translate business requirements into data solutions; collaborate with teams.

Responsibilities

  • Design, build and maintain production ETL/ELT pipelines using Databricks, PySpark, SQL and Delta Lake.
  • Integrate data from databases, APIs and files with streaming ingestion.
  • Develop curated datasets and data models per business definitions and access needs.
  • Implement data quality checks, reconciliation, incremental processing, schema handling and recovery.
  • Monitor pipeline reliability and data freshness; investigate failures and root causes.
  • Optimize processing performance and job compute usage with Platform Engineer.
  • Apply version control, automated testing, CI/CD and documentation.
  • Automate repetitive tasks; assess AI-assisted approaches vs accuracy and effort saved.

Skills

Python
SQL
Spark / PySpark
Data modelling
Collaboration with cross-functional

Tools

Databricks
Delta Lake
Git
CI/CD

Job description

Singapore, Singapore

Senior Data Engineer Databricks and Automation

Data and AI Innovation Team

About the role

Build reliable data pipelines and trusted datasets for analytics, business applications and AI use cases. This hands-on role centres on Databricks, Python, PySpark and SQL. You will also simplify recurring engineering and support activities through automation, using AI where it provides a clear benefit.

Key responsibilities
  • Design, build and maintain production ETL/ELT pipelines using Databricks, PySpark, SQL and Delta Lake.
  • Integrate data from databases, APIs and files, with streaming ingestion where required.
  • Develop curated datasets and data models that meet agreed business definitions, access requirements and consumption needs.
  • Implement data quality checks, reconciliation, incremental processing, schema handling and recovery procedures.
  • Monitor pipeline reliability and data freshness; investigate failures, resolve root causes and improve recovery.
  • Optimise processing performance and job compute usage, partnering with the Platform Engineer on shared settings and controls.
  • Apply version control, automated testing, CI/CD and documentation to support reliable releases and maintainable solutions.
  • Automate repetitive engineering and support tasks; assess AI-assisted approaches against accuracy, security and manual effort saved.
Essential skills and experience
  • Typically 5 or more years of hands-on data engineering experience, including building and supporting production pipelines.
  • Strong Python, SQL and Spark or PySpark skills.
  • Practical Databricks and Delta Lake experience, including job orchestration and troubleshooting. Strong production Spark and lakehouse experience with less Databricks exposure will also be considered.
  • Experience with data modelling, incremental ingestion, data quality, reconciliation and performance tuning.
  • Experience with a cloud data platform and practical use of Git, testing and deployment pipelines.
  • Ability to translate business requirements into maintainable data solutions and collaborate with business and technical teams.
Preferred experience
  • Azure data services and enterprise data integration.
  • Unity Catalog, Auto Loader, Structured Streaming or Databricks deployment tooling.
  • Streaming technologies such as Kafka or Event Hubs.
  • Automation of monitoring, reconciliation or operational support.
  • Practical use of LLMs, AI agents or Databricks AI capabilities to improve engineering workflows.
Role ownership

Own data pipelines, data quality, curated datasets and job-level performance. Partner with the Platform Engineer on shared environments, deployment tooling, access controls and platform monitoring.

What success looks like

Accurate and timely data, reliable pipelines, efficient processing, faster recovery from failures and less manual support effort.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer – AWS / Databricks / PySpark
Senior Data Engineer – AWS / Databricks / PySpark

ASTEK SINGAPORE INNOVATION TECHNOLOGY PTE. LTD. • Singapore

On-site
SGD 120,000 - 180,000
Senior Data Engineer – AWS / Databricks / PySpark
Senior Data Engineer – AWS / Databricks / PySpark

Astek Group • Singapore

On-site
SGD 120,000 - 180,000
Senior Data Engineer AWS / Databricks / PySpark
Senior Data Engineer AWS / Databricks / PySpark

astek singapore innovation technology pte. ltd. • Singapore

On-site
SGD 120,000 - 190,000
Senior Data Engineer (AWS & Databricks) | Contract
Senior Data Engineer (AWS & Databricks) | Contract

nexbridge recruitment pte. ltd. • Singapore

On-site
SGD 90,000 - 180,000
Senior Data Engineer | Contract | Hybrid Working Arrangement
Senior Data Engineer | Contract | Hybrid Working Arrangement

NEXBRIDGE RECRUITMENT PTE. LTD. • Singapore

On-site
SGD 120,000 - 180,000
Senior Data Engineer
Senior Data Engineer

ExcelTech Computers Pte. Ltd. • Singapore

On-site
SGD 120,000 - 180,000
AI-Driven Data Engineer: Databricks + AWS Pipelines
AI-Driven Data Engineer: Databricks + AWS Pipelines

PERSOL • Singapore

On-site
SGD 90,000 - 130,000
Senior Data Engineer
Senior Data Engineer

zhong qi future (singapore) pte. ltd. • Singapore

Hybrid
SGD 120,000 - 180,000
M30 - Data Engineer
M30 - Data Engineer

FPT Asia Pacific • Singapore

On-site
SGD 120,000 - 180,000
Data Engineer (Databricks)
Data Engineer (Databricks)

PERCEPT SOLUTIONS PTE. LTD. • Singapore

On-site
SGD 90,000 - 130,000