Data Engineer: Cloud Pipelines, PySpark & Iceberg

Aryan-Solutions-Pte.-Ltd.; Singapore; On-site; SGD 80,000 - 120,000; Full time

Job summary

Aryan-Solutions-Pte.-Ltd. is seeking a data engineer to build and maintain data pipelines using cloud technologies and distributed systems.

The role involves working with data transformation frameworks and orchestration tools, and managing large data volumes. The ideal candidate will have hands-on Python experience, plus formal training in data engineering concepts and familiarity with AWS, SQL, PySpark, Apache Iceberg, dbt and Airflow.

Qualifications

• Formal training or certification in data engineering concepts is required.
• Recent hands-on professional experience as a data engineer.
• Experience coding in Python.
• Hands-on experience with AWS, SQL, PySpark, Iceberg, dbt and Airflow.
• Experience with cloud technologies and distributed systems.

Responsibilities

• Build and maintain data pipelines using cloud technologies and distributed systems.
• Work with data transformation frameworks and pipeline orchestration tools.
• Manage large volumes of data and optimise data processing.
• Design and implement effective tests, including unit, component, integration, end-to-end and performance tests.
• Coach team members on coding practices, design principles and implementation patterns.
• Manage stakeholders and prioritise work effectively across multiple workstreams.

Education

Formal training or certification in data engineering concepts

Tools

AWS

SQL

PySpark

Apache Iceberg

dbt

Apache Airflow

Job description

Aryan-Solutions-Pte.-Ltd. is seeking a data engineer to build and maintain data pipelines using cloud technologies and distributed systems.

The role involves working with data transformation frameworks and orchestration tools, and managing large data volumes. The ideal candidate will have hands-on Python experience, plus formal training in data engineering concepts and familiarity with AWS, SQL, PySpark, Apache Iceberg, dbt and Airflow.