Data Engineer - Databricks & Automation

U3 INFOTECH PTE. LTD.

Singapore

On-site

SGD 90,000 - 150,000

Full time

2 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

U3 INFOTECH PTE. LTD. is seeking a data engineer to design and maintain scalable ETL/ELT pipelines using Databricks, PySpark, SQL and Delta Lake.

The role focuses on integrating data from databases, APIs and files, with streaming where needed, and building curated datasets for reporting, analytics and AI applications. You will implement data quality checks, monitor pipelines, troubleshoot issues and optimise performance, while applying CI/CD and technical documentation practices to support

Responsibilities

  • Design, develop and maintain scalable ETL/ELT data pipelines using Databricks, Python, PySpark, SQL and Delta Lake.
  • Integrate data from multiple sources, including databases, APIs and files, with streaming ingestion where required.
  • Develop and maintain curated datasets and data models to support business reporting, analytics and AI applications.
  • Implement data quality checks, data validation, reconciliation and incremental data processing.
  • Monitor data pipelines, investigate failures, troubleshoot issues and ensure data accuracy and availability.
  • Optimise data processing performance, pipeline execution and compute utilisation.
  • Implement version control, automated testing, CI/CD and technical documentation.
  • Identify and automate repetitive data engineering and support activities.
  • Explore practical AI

Job description

Key Responsibilities
  • Design, develop and maintain scalable ETL/ELT data pipelines using Databricks, Python, PySpark, SQL and Delta Lake.

  • Integrate data from multiple sources, including databases, APIs and files, with streaming ingestion where required.

  • Develop and maintain curated datasets and data models to support business reporting, analytics and AI applications.

  • Implement data quality checks, data validation, reconciliation and incremental data processing.

  • Monitor data pipelines, investigate failures, troubleshoot issues and ensure data accuracy and availability.

  • Optimise data processing performance, pipeline execution and compute utilisation.

  • Implement version control, automated testing, CI/CD and technical documentation.

  • Identify and automate repetitive data engineering and support activities.

  • Explore practical AI

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

Carecone Group • Singapore

On-site
SGD 100,000 - 180,000
Databricks Data Engineer – Automation & AI Pipelines
Databricks Data Engineer – Automation & AI Pipelines

U3 INFOTECH PTE. LTD. • Singapore

On-site
SGD 90,000 - 150,000
Data Engineer, AI Integration Specialist
Data Engineer, AI Integration Specialist

PERSOL • Singapore

On-site
SGD 90,000 - 130,000
Data Engineer (Databricks)
Data Engineer (Databricks)

NCS Pte Ltd • Singapore

On-site
SGD 90,000 - 150,000
Data Engineer (Databricks & DevOps)
Data Engineer (Databricks & DevOps)

COGNIZANT TECHNOLOGY SOLUTIONS ASIA PACIFIC PTE. LTD. • Singapore

On-site
SGD 90,000 - 150,000
Databricks Workflows
CloudWatch Monitoring
Data Engineer
Data Engineer

EPS COMPUTER SYSTEMS PTE LTD • Singapore

On-site
SGD 90,000 - 150,000
Data Engineer
Data Engineer

User Experience Researchers Pte.Ltd • Singapore

On-site
SGD 90,000 - 150,000
Senior Data Engineer, Databricks & AI Automation
Senior Data Engineer, Databricks & AI Automation

Flintex Consulting Pte Ltd • Singapore

On-site
SGD 120,000 - 180,000
Data Engineer (Databricks)
Data Engineer (Databricks)

PERCEPT SOLUTIONS PTE. LTD. • Singapore

On-site
SGD 90,000 - 130,000
Data Engineer - Databricks & AWS Data Pipelines
Data Engineer - Databricks & AWS Data Pipelines

EPS COMPUTER SYSTEMS PTE LTD • Singapore

On-site
SGD 90,000 - 150,000