Technical Architect - Big Data

VUI SYSTEMS PTE. LTD.; Singapore; On-site; SGD 60,000 - 90,000; Full time

Job summary

A leading data solutions company in Singapore is seeking an experienced Data Architect to define reusable design patterns, provide technical leadership, and optimize development of big data solutions. The ideal candidate should possess strong programming abilities in Python and expertise in cloud services like AWS or Azure. Key responsibilities include participating in design reviews, developing foundational components, and evaluating methodologies. This role demands a strong affinity for IT solutions and a track record in data management.

Qualifications

• Strong affinity for IT solutions and data architecture.
• Experience in delivering big data solutions, including data modeling.
• Hands-on experience with cloud services and big data programming.

Responsibilities

• Define reusable design patterns for data processes.
• Provide technical leadership and participate in design/code reviews.
• Optimize development through solution accelerators.

Skills

Data architecture

Big Data solutions

Data modeling

Python

Cloud services

Tools

AWS

Azure

GCC

PySpark

Kafka

Job description

Key Competencies

· Define reusable design patterns e.g. data movement (in/outbound) processes, data validation, data load & extraction processes etc.

· Create re-usable connections to different targets (DB/Hadoop etc.)

· Setup metadata repository for connections, schemas and contexts to be used across jobs

· Define / Develop foundational components - Error / Audit framework, Source acquisition ETL frame work, etc

· Evaluate and recommend the development methodologies, frameworks and modelling techniques.

· Provide technical leadership to the development team in preparing the design artifacts and implementation of integrations.

· Participate in design/code reviews and ensure that all solutions are aligned to pre-defined architectural specifications

· Optimize development through solution accelerators & value creators.

Experience

· Strong affinity for IT solutions and data architecture on macro as well as on micro level

· Conceptual brilliance with a track record for (big) data solutions incl. data modeling (structured/unstructured data)

· Sound experience in driving and delivering change

Beside data management and modeling, the architect will need to be familiar or have prior experience with other analytics area, such a dashboarding.

• Hands on Experience in Design and Development on Big Data Platform (GCC or AWS or Azure etc.)
• Hands on programming experience on Big Data (Python, PySpark, Spark Scala etc.)
• Hand on Experience in cloud services (Kinesis, Kafka, Lambda, Databricks etc.)
• Hands on experience on Storage covering Schema Layout, data modeling, partitioning, RDBMS, (Hive, HBase, MongoDB , Mysql, Sql, Casandra, S3, Dynamo DB, Redshift etc.)
• Expertise in understanding client business and providing solution architecture in cloud covering Data Ingestion, Storage and Visualization