Senior Data Engineer - Databricks & Real-Time Streaming

PERSOL SINGAPORE PTE. LTD.

Singapore

On-site

SGD 90,000 - 130,000

Full time

14 days+

Application generator

Don’t send a generic resume — generate a resume and cover letter tailored to this exact role.

Get past ATS filters

Job summary

PERSOL Singapore PTE. LTD. is seeking an experienced Data Engineer to develop and maintain data processing solutions using Databricks and Python, and to design real-time streaming pipelines with Apache Spark Structured Streaming.

You will troubleshoot and optimize streaming jobs, work with large data volumes, and collaborate with data engineers and stakeholders to deliver effective solutions while following best practices for performance and maintainability.

Qualifications

5–10 years of overall experience in data engineering / data development.
Strong hands-on experience with Databricks.
Strong experience with Apache Spark Structured Streaming.
Strong Python programming skills with a good understanding of data processing concepts.

Responsibilities

Develop and maintain data processing solutions using Databricks and Python.
Design and implement real-time data processing pipelines using Apache Spark Structured Streaming.
Develop efficient and scalable data processing logic using strong Python programming fundamentals and data structures.
Troubleshoot and optimize streaming jobs and data processing workflows.
Work with large volumes of data and ensure reliable, scalable, and efficient processing.
Monitor and resolve issues related to streaming pipelines and Databricks workloads.
Collaborate with data engineers, technical teams, and stakeholders to understand requirements and deliver effective solutions.
Follow best practices for code quality, performance, maintainability, and operational support.
Participate in troubleshooting, debugging, and performance optimization of data pipelines.