Backend Data Engineer

PERSOL SINGAPORE PTE. LTD.

Singapore

On-site

SGD 70,000 - 110,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

PERSOL SINGAPORE PTE. LTD. is seeking a Data Engineer to design, implement, and maintain data pipelines for semiconductor verification outputs.

You will work with web developers and DevOps to automate collection, processing, and visualization of run results, enabling smarter decision making and faster chip tapeouts. The role requires strong Python data processing skills and hands-on experience with Apache Spark, Airflow, Kafka, and ClickHouse, along with containerization and monitoring tools.

Qualifications

  • Bachelor’s degree or higher in Computer Science, Electrical Engineering, or related field.
  • Strong programming skills in Python for data processing.
  • Deep hands-on experience in Big Data Frameworks, e.g. Apache Spark, Airflow, Kafka, ClickHouse.

Responsibilities

  • Develop scalable data pipelines to ingest, parse, and aggregate data from EDA tools and Mongo databases.
  • Collaborate with web teams to integrate processed data into dashboards or web-based reporting tools.
  • Design and manage databases for tracking job metrics, violation summaries, and run metadata.
  • Automate reporting and alerting systems for regressions, failure trends, and job status.
  • Optimize storage and retrieval of large log files, job results, and associated metadata.
  • Maintain and improve log parsing scripts.

Skills

Python
Data processing
Big data concepts

Education

Bachelor's degree in CS/EE

Tools

Apache Spark
Airflow
Kafka
ClickHouse
Docker/Podman
Prometheus
Grafana
ELK Stack

Job description

Job Description

As a Data Engineer in our web development team supporting chip verification, you will design, implement, and maintain data pipelines that manage data and metrics from physical verification outputs in the semiconductor design process. You will work closely with web developers and DevOps teams to automate collection, processing, and visualization of run results, empowering smarter decision-making and accelerating our chip tapeouts.

Responsibilities
  • Develop scalable data pipelines to ingest, parse, and aggregate data from EDA tools and Mongo databases.

  • Collaborate with web teams to integrate processed data into dashboards or web-based reporting tools.

  • Design and manage databases for tracking job metrics, violation summaries, and run metadata.

  • Automate reporting and alerting systems for regressions, failure trends, and job status.

  • Optimize storage and retrieval of large log files, job results, and associated metadata.

  • Maintain and improve log parsing scripts.

Requirements
  • Bachelor’s degree or higher in Computer Science, Electrical Engineering, or related field.

  • Strong programming skills in Python for data processing.

  • Deep hands-on experience in Big Data Frameworks, e.g. Apache Spark, Airflow, Kafka, ClickHouse.

Other Preferred Skills
  • Experience with containerization (Docker/Podman), monitoring tools (Prometheus, Grafana, ELK Stack), and CI/CD pipelines.

  • Ability to quickly prototype and iterate on solutions.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

Horizon Computer Management • Singapore

On-site
SGD 120,000 - 160,000
Data Engineer
Data Engineer

PageGroup Corporation Solutions (Malaysia) Sdn Bhd • Singapore

On-site
SGD 80,000 - 120,000
Data Engineer
Data Engineer

EPS COMPUTER SYSTEMS PTE LTD • Singapore

On-site
SGD 90,000 - 150,000
Data Engineer
Data Engineer

TALENTSIS PTE. LTD. • Singapore

On-site
SGD 60,000 - 110,000
Data Engineer/ Senior Data Engineer
Data Engineer/ Senior Data Engineer

Softvil Technologies Pvt Ltd • Singapore

On-site
SGD 60,000 - 80,000
Data Engineer (Python / SQL / AWS) #10480
Data Engineer (Python / SQL / AWS) #10480

ANRADUS PTE. LTD. • Singapore

On-site
SGD 60,000 - 90,000
Senior Data Engineer
Senior Data Engineer

Epergne Solutions • Singapore

On-site
SGD 120,000 - 180,000
Data Engineer
Data Engineer

TALENTVIS SINGAPORE PTE. LTD. • Singapore

On-site
SGD 90,000 - 120,000
Data Engineer (Big Data)
Data Engineer (Big Data)

Spearing Executive Search • Singapore

On-site
SGD 70,000 - 90,000
Data Engineer
Data Engineer

HCL SINGAPORE PTE. LTD. • Singapore

On-site
SGD 120,000 - 180,000