Data Engineer

TOPPAN ECQUARIA PTE. LTD.; Singapore; In loco; SGD 90.000 - 130.000; Tempo pieno

Descrizione del lavoro

TOPPAN ECQUARIA PTE. LTD. is seeking a data engineering professional to design, build and maintain large-scale batch and real-time data pipelines in Singapore.

You will ingest, clean and transform data using Spark and big data frameworks, and design data tables in data lakes and data warehouses to support analytics. Responsibilities include developing backend APIs and integrating databases to support applications, collaborating with Agile teams, and ensuring data governance and security best

Competenze

• Degree or Diploma in CS/CE/IT/Business Analytics or related.
• 2–8 years' experience designing, developing or maintaining data platforms.
• Led a team of data engineers.
• Experience with Spark, Scala or PySpark.
• Experience with Hadoop, Azure Data Factory, Azure Synapse.
• Data modeling/design and management (Data Mart, Data Lake, Data Warehousing).
• Experience building production-grade data pipelines using cloud platforms such as Azure.
• Experience with Agile teams.
• Knowledge of data governance, access control and security best practices.
• Stakeholder engagement/requirements analysis.

Mansioni

• Design, build and maintain efficient and reliable large-scale batch and real-time data pipelines.
• Ingest, clean, transform data to ensure pipelines are efficient, reliable and scalable.
• Design, develop and deploy data tables in data lake/data warehouses.
• Design and develop backend APIs and database integrations to support applications/use cases.

Conoscenze

Technical leadership

Agile methodologies

Stakeholder engagement

Formazione

Degree or Diploma in CS/CE/IT/Business Analytics

Strumenti

Spark

Scala/PySpark

Hadoop

Azure Data Factory

Azure Synapse

Data Lake

Descrizione del lavoro

Design, build and maintain efficient and reliable large-scale batch and real-time data pipelines

Utilize data processing/engineering frameworks for data ingestion, cleaning, transformation to ensure the data pipelines are efficient, reliable and scalable

Design, develop and deploy data tables in data lake / data warehouses.

Design and develop backend APIs and database integrations to support applications/use cases.

Prerequisites

• Degree or Diploma in Computer Science, Computer Engineering, Information Technology, Business Analytics or related disciplines.
• Minimum 2 - 8 years' experience in designing, developing and/or maintaining data platforms.
• Played the role of a technical lead to a team of data engineers.
• Spark, Scala/Pyspark
• Big data frameworks/tools (e.g. Hadoop, Azure Data Factory, Azure Synapse)
• Data platform and data structure/model design and management (e.g. Data Mart, DataLake, Data Virtualization and Data Warehousing)
• Experience with building production-grade data pipelines, ETL/ELT data integration in a big data environment using Cloud platforms such as Azure.
• Experience working with Agile teams with strong collaboration skills.
• Knowledge/experience on data governance policies, access control and security best practices.
• Stakeholders' engagement/management and requirements analysis skills.