Solution Architect

UNISYNC SYSTEMS PTE. LTD.

Dublin

Vor Ort

EUR 90.000 - 120.000

Vollzeit

Vor 8 Tagen

Erhalte eine Antwort von diesem Arbeitgeber

Dein Lebenslauf und dein Anschreiben, genau auf das zugeschnitten, was gesucht wird.

Schaffe es an den ATS-Filtern vorbei

Zusammenfassung

UNISYNC SYSTEMS PTE. LTD. is seeking an experienced Data Engineer to design and operate scalable data pipelines within an enterprise Data Lake, leveraging Cloudera CDP, Hadoop, Spark, Hive, Java, Python, and SQL.

The role focuses on batch and near real-time ingestion to onboard diverse enterprise sources into the Lake. You will drive data quality and governance while collaborating with platform teams to optimize cluster utilization.

Qualifikationen

  • 10+ years of experience in data engineering or related field.
  • 5+ years managing Enterprise Data Lake/ DW or Big Data programs.
  • Experience in Hadoop/Cloudera environments is required.
  • Banking/Financial Services experience preferred.

Aufgaben

  • Design, develop and maintain scalable data pipelines in Enterprise Data Lake using Cloudera CDP/Hadoop/Spark/Hive/SQL.
  • Develop high-performance batch and real-time data ingestion for onboarding source systems.
  • Implement data transformations using Spark, HiveQL, SQL and Java-based processing.
  • Design and implement ingestion patterns for structured, semi-structured and unstructured data.
  • Build and maintain batch/near real-time data integration solutions.
  • Define source-to-target mappings and ingestion specs.

Kenntnisse

Stakeholder mgmt
Project delivery
Data engineering leadership
BAU portfolio mgmt

Tools

Hadoop
Cloudera CDP
Spark
Hive
Java
Python
SQL
Kafka
YARN/Oozie
Unix/Linux scripting
Shell scripting

Jobbeschreibung

This role requires strong technical knowledge of enterprise data platforms, excellent stakeholder management skills, and proven experience delivering complex projects within the banking and financial services industry.

  • 10+ years of experience.
  • 5+ years managing Enterprise Data Lake, Data Warehouse or Big Data delivery initiatives.
  • Proven experience in Hadoop/Cloudera environments.
  • Experience managing multiple concurrent project deliveries and BAU portfolios.
  • Banking and Financial Services experience preferred.
Data Engineering & Integration
  • Design, develop, and maintain scalable data pipelines within the Enterprise Data Lake platform using Cloudera CDP, Hadoop, Spark, Hive, Java, Python, and SQL.
  • Develop high-performance batch and near real-time data ingestion frameworks for onboarding enterprise source systems into the Data Lake.
  • Develop complex transformation logic using Spark, HiveQL, SQL, and Java-based processing frameworks.
  • Design and implement data ingestion patterns supporting structured, semi-structured, and unstructured datasets.
  • Build and maintain batch and near real-time data integration solutions.
  • Implement source-to-target mappings and ingestion specifications.
Hadoop & Cloudera Platform Development
  • Develop and support solutions within the Cloudera Data Platform ecosystem.

Build and maintain data pipelines leveraging:

  • Hadoop HDFS
  • Hive
  • Impala
  • Spark
  • Kafka
  • YARN, Oozie
  • Perform deployment, troubleshooting, and performance tuning of applications running on the Hadoop ecosystem.
  • Partner with platform teams to ensure optimal cluster utilization and workload management.
Data Quality & Governance
  • Ensure adherence to Data Lake onboarding standards and governance requirements.
  • Investigate and resolve data quality issues and production incidents.
  • Ensure compliance with enterprise data governance standards.
DevOps & Platform Automation
  • Integrate data engineering solutions into CI/CD pipelines using Git, Jenkins, Azure DevOps, or equivalent tools.
  • Automate deployment, testing, monitoring, and operational support activities.
  • Develop reusable frameworks and utilities to improve engineering productivity and platform scalability.
Production Support & Incident Management
  • Provide L3 support for Data Lake application.
  • Perform root cause analysis for production incidents and implement permanent fixes.
  • Monitor pipeline execution, cluster health, and workload performance.
  • Participate in release implementation, deployment validation, and post-production support activities.
  • Strong hands on experience with Hadoop ecosystem and data pipeline orchestration.
  • Hadoop/HDFS
  • Hive
  • Spark
  • Unix/Linux scripting
  • Shell scripting
  • ETL/ELT development experience.
  • Data modelling fundamentals.
  • Performance tuning and optimization.
  • Experience in Data Warehousing and Data Lake environments.
  • Strong Java / Python competency with experiences.
  • Strong SQL development skills.
Hol dir deinen kostenlosen, vertraulichen Lebenslauf-Check.

oder ziehe deine Datei hierhin.

Similar jobs

Ähnliche Jobs, die dir auch gefallen könnten

APPLICATIONS DEVELOPER
APPLICATIONS DEVELOPER

UNISYNC SYSTEMS PTE. LTD. • Dublin

Vor Ort
EUR 90.000 - 130.000
Data Engineer
Data Engineer

ACCENTURE PTE LTD • Dublin

Vor Ort
EUR 90.000 - 120.000
Senior Data Engineer
Senior Data Engineer

PlayStation • Irland

Vor Ort
EUR 90.000 - 130.000
Senior Data Platform Architect – Banking & Big Data
Senior Data Platform Architect – Banking & Big Data

UNISYNC SYSTEMS PTE. LTD. • Dublin

Vor Ort
EUR 90.000 - 120.000
Senior Data Engineer - Lakehouse Pipelines & AI
Senior Data Engineer - Lakehouse Pipelines & AI

G MASS Consulting • Dublin

Vor Ort
EUR 90.000 - 130.000
Data Engineer - Python / Databricks
Data Engineer - Python / Databricks

Cpl Group • Dublin

Vor Ort
EUR 110.000 - 160.000
Senior Data Engineer
Senior Data Engineer

G MASS Consulting • Dublin

Vor Ort
EUR 90.000 - 130.000
Senior Data Engineer - Databricks Lakehouse & AI Pipelines
Senior Data Engineer - Databricks Lakehouse & AI Pipelines

Expleo Group • Dublin

Vor Ort
EUR 90.000 - 140.000
Senior Data Engineer with Manufacturing
Senior Data Engineer with Manufacturing

Intuition IT Solutions • Irland

Vor Ort
EUR 90.000 - 130.000
Senior Engineer (R-20003)
Senior Engineer (R-20003)

Dun & Bradstreet • Dublin

Vor Ort
EUR 120.000 - 180.000
Top-Standorte