Data Engineer

IOTALENTS PTE. LTD.; Singapore; On-site; SGD 65,000 - 85,000; Full time

Job summary

A data solutions company in Singapore is looking for an experienced data engineer to develop and support data ingestion pipelines. The ideal candidate will have at least 3 years of experience, proficiency in SQL and Python, and familiarity with cloud environments such as AWS. You will work closely with data teams to ensure data quality and compliance. This position offers an opportunity to work in a dynamic environment where learning and adaptability are key.

Qualifications

• At least 3 years of experience in data ingestion pipelines.
• Experience with data warehousing concepts.
• Familiarity with SDLC methodology.

Responsibilities

• Work with data stewards and analysts to implement data solutions.
• Design scalable data management solutions using Python.
• Ensure compliance with IT security standards.

Skills

Data ingestion pipeline development

SQL proficiency

Python proficiency

Problem-solving skills

Communication skills

Agile adaptation

Education

Degree in Computer Science/Information Technology

Tools

Snowflake

AWS

GitHub

Job description

Responsibilities

• Work closely with data stewards, data analysts, and business end-users to implement and support data solutions.
• Design and build robust and scalable data ingestion and data management solutions for batch-loading and streaming from multiple data sources using Python via different mechanisms such as API, file transfer, direct interface with Oracle and MSSQL databases.
• Familiar with SDLC process: Requirement gathering, design and development, SIT testing, supporting UAT, and CICD deployment using GitHub for enhancement and a new ingestion pipeline.
• Ensure compliance with IT security standards, policies, and procedures. Provide BAU support in terms of production job monitoring, issue resolution, and bug fixes.
• Enable ingestion checks and data quality checks for all data sets in the data platform and ensure the data issues are actively detected, tracked, and fixed without breaching SLA.

Requirements

• Possess a degree in Computer Science/Information Technology or related fields.
• At least 3 years of experience in a role focusing on the development and support of data ingestion pipelines.
• Experience with building on data platforms, e.g., Snowflake.
• Proficient in SQL and Python.
• Experience with Cloud environments (e.g., AWS).
• Experience with continuous integration and continuous deployment (CICD) using GitHub.
• Experience with Software Development Life Cycle (SDLC) methodology.
• Experience with data warehousing concepts.
• Strong problem-solving and troubleshooting skills.
• Strong communication and collaboration skills.
• Able to design and implement solutions and perform code review independently.
• Able to provide production support independently.
• Agile, fast learner, and able to adapt to changes.