Date Engineer

ABS CONNECT PTE. LTD.; Serangoon Garden Circus; On-site; SGD 80,000 - 120,000; Full time

Job summary

A technology solutions company based in Singapore is seeking an experienced Data Engineer. The role involves developing large-scale data pipelines utilizing Big Data technologies, particularly PySpark and Python. Candidates should have 4-10 years of experience, strong SQL skills, and familiarity with Agile methodologies. This position offers the opportunity to work closely with business users and stakeholders to translate requirements into effective technical solutions.

Qualifications

• 4-10 years of experience as a Data Engineer in enterprise-level data platforms.
• Proven expertise in designing, developing, and maintaining scalable Big Data pipelines.
• Strong debugging and analytical skills for addressing complex data issues.

Responsibilities

• Interact directly with business users and stakeholders for data solutions.
• Participate in end-to-end development and implementation activities.
• Implement CI/CD pipelines and participate in Agile sprints.

Skills

Big Data technologies

PySpark

Python

SQL

DevOps

Agile methodologies

Git

Job description

Must Have

• Proven experience in Data Engineering as an individual contributor, including direct interactions with business users, cross‑functional stakeholders, and participation in end‑to‑end development and implementation activities.
• Strong hands‑on expertise with Big Data technologies, PySpark, Python, DevOps practices, and performance tuning of large‑scale data workloads.
• High proficiency in SQL along with a solid understanding of data modeling concepts and best practices.
• Working knowledge of version control systems (Git), DevOps CI/CD pipelines, and Agile development methodologies.
• Deep understanding of distributed computing fundamentals, including RDDs, Data Frames, partitioning strategies, and cluster execution behavior.
• Excellent debugging, analytical, and problem‑solving skills with the ability to troubleshoot complex data issues
  

Experience

• 4–10 years of experience as a Data Engineer working in enterprise‑level data platforms.
• Proven expertise in designing, developing, and maintaining scalable Big Data pipelines using PySpark and Python.
• Hands‑on experience working with business users and cross‑functional teams to convert business requirements into robust technical solutions.
• Strong proficiency in SQL with extensive experience in query optimization and relational data modeling.
• Experience implementing CI/CD pipelines, Git‑based version control, and DevOps best practices.
• Worked within Agile environments, participating in sprints, requirement grooming, and technical reviews.
• Expertise in performance tuning, including Spark job optimization, partitioning strategies, and efficient resource utilization.
• Strong debugging and analytical skills for addressing complex data issues and system inconsistencies.
  

Good‑to‑Have Skills

• Additional expertise in advanced data modeling practices (conceptual, logical, and physical modeling).
• Experience with PySpark code migration, including refactoring, platform upgrades, and performance enhancements.