Principal Consultant

HTC SOFTWARE SINGAPORE PTE LTD; Singapore; On-site; SGD 80,000 - 130,000; Full time

Job summary

A leading company is seeking a Principal Consultant Data Engineer for its Singapore operations. The role involves developing data pipelines, requiring extensive experience in Spark, Trino, DBT, and Iceberg technology. Candidates should be eager to learn and adapt to new technologies and possess excellent communication skills.

Qualifications

• Minimum 5 years of experience in Spark, DBT, Trino and Iceberg technology.
• Knowledge or working experience in the banking sector preferred.
• Eagerness to learn and adapt to new technologies in Data & AI.

Responsibilities

• Developed data pipelines to extract, transform, and load Core Motion data.
• Processed data frames during data wrangling with SQL through Spark.
• Architecture level experience preferred with hands-on coding.

Skills

Coding in Spark

DBT

Trino

Iceberg technology

Data wrangling

Excellent verbal skills

Excellent written skills

Education

Azure Solutions Architect Expert

AWS Certified Data Engineer

Salesforce Platform Developer II

Salesforce Consultant Certificates

Tools

Gitlab

Job description

We are looking for Principal Consultant Data Engineer in Singapore. Must have Coding experience in Spark, DBT, Trino and Iceberg technology. Architecture level experience will be preferred with hands-on coding.

· Minimum 5 years of experience in Spark, DBT, Trino and Iceberg technology

· Developed on data pipeline to periodically extract, transform, and load Core Motion data from different types of raw data (parquet, csv, etc.) into parquet files using Python, Spark, and Pandas with qualified data for future analysis.

· Processed data frames during data wrangling with SQL statements through Spark based

· The following additional certificates will be an added advantage: Azure Solutions Architect Expert, AWS Certified Data Engineer, Salesforce Platform Developer II, Salesforce Consultant Certificates.

· Knowledge or understanding of project lifecycle.

· Knowledge or working experience in banking sector is preferred.

· Experience with integration on relevant tools and software such as Spark, DBT, Trino and Iceberg, Gitlab.

· Eagerness to learn and adapt to new technologies especially in the Data & AI domain.

· Excellent verbal and written skills.