Senior Enterprise Data Architect - Hadoop & Data Lake

UNISYNC SYSTEMS PTE. LTD.; Singapore; On-site; SGD 180,000 - 240,000; Full time

Job summary

UNISYNC SYSTEMS PTE. LTD. is seeking an experienced Data Engineer to design, build and maintain scalable data pipelines within an Enterprise Data Lake. You will work on batch and near real-time ingestion, data transformation, and integration using Hadoop, Spark, Hive, Java and Python.

The role requires 10+ years in data engineering and 5+ years managing complex Data Lake/Warehouse initiatives, preferably in Banking/Financial Services. Cloud/Data Platform experience desirable.

Qualifications

• 10+ years of experience in data engineering or related field.
• 5+ years managing Enterprise Data Lake, Data Warehouse or Big Data delivery initiatives.
• Proven experience in Hadoop/Cloudera environments.
• Experience managing multiple concurrent project deliveries and BAU portfolios.
• Banking and Financial Services experience preferred.

Responsibilities

• Design, develop, and maintain scalable data pipelines in the Data Lake platform using Cloudera CDP, Hadoop, Spark, Hive, Java, Python, and SQL.
• Develop high-performance batch and near real-time data ingestion frameworks for onboarding enterprise source systems into the Data Lake.
• Develop complex transformation logic using Spark, HiveQL, SQL, and Java-based processing frameworks.
• Design and implement data ingestion patterns for structured, semi-structured, and unstructured datasets.
• Build and maintain batch and near real-time data integration solutions.
• Implement source-to-target mappings and ingestion specifications.
• Develop and support solutions within the Cloudera Data Platform ecosystem.
• Perform deployment, troubleshooting, and performance tuning of Hadoop applications.
• Partner with platform teams to ensure optimal cluster utilization and workload management.
• Ensure adherence to Data Lake onboarding standards and governance requirements.
• Investigate and resolve data quality issues and production incidents.
• Ensure compliance with enterprise data governance standards.
• Integrate data engineering solutions into CI/CD pipelines using Git, Jenkins, Azure DevOps, or equivalent tools.
• Automate deployment, testing, monitoring, and operational support activities.
• Develop reusable frameworks and utilities to improve engineering productivity and platform scalability.
• Provide L3 production support for Data Lake applications and perform root cause analysis.

Skills

Data engineering

Stakeholder management

Project delivery management

Hadoop/Cloudera

Spark

Python

Java

SQL

Data pipelines

Data governance

Tools

Hadoop HDFS

Hive

Impala

Spark

Kafka

YARN

Oozie

Git

Jenkins

Azure DevOps

Job description

UNISYNC SYSTEMS PTE. LTD. is seeking an experienced Data Engineer to design, build and maintain scalable data pipelines within an Enterprise Data Lake. You will work on batch and near real-time ingestion, data transformation, and integration using Hadoop, Spark, Hive, Java and Python.

The role requires 10+ years in data engineering and 5+ years managing complex Data Lake/Warehouse initiatives, preferably in Banking/Financial Services. Cloud/Data Platform experience desirable.