Senior Data Engineer (Big Data & Data Warehouse)

Caixa Mágica Software; Lisbon (AR); On-site; USD 90,000 - 130,000; Full time

Benefits offered by this job

Health and Life Insurance

Permanent contract

Training in latest technologies

Team events

Tech equipment + SIM Card + smartphone

Office coffee, fruits and snacks

Job summary

Caixa Mágica Software is seeking a seasoned data/ETL analyst to design, implement and test IT solutions within Data Warehouse projects. You will formalise analyses, document solutions and participate in stakeholder meetings, ensuring alignment with the roadmap.

The role involves certifying quality tests, improving platforms and monitoring UATs for production reliability. We value >6 years in service environments, expertise in SQL/NoSQL, Hadoop, Kafka, and data modelling, with Agile experience

Qualifications

• Experience in the service environment: >6 years.
• Ingestion of raw data pipelines (Data Lake or similar).
• Knowledge of data access, referencing and quality.
• Expertise in Hadoop ecosystem and data management.
• DevOps, data environment mastery, and data governance.
• Experience with Agile projects and banking domain preferred.

Responsibilities

• Analyze, design, implement and test IT solutions within DataWarehouse projects.
• Document technical analyses, solutions, tests and decisions.
• Lead meetings with stakeholders and oversee UAT and certification tests.
• Propose improvements to management platforms and tools to optimise systems.
• Ensure production environment reliability and monitor service commitments.

Skills

SQL

NoSQL

Java

Python

Hive

Spark

ETL

Bash

Power BI

Kafka

Nifi

Hadoop ecosystem

DevOps

Education

Bachelor's or Master's degree in Computer Engineering, Information Technology or similar areas

Tools

Jira

Confluence

Draw IO

Job description

Overview:

Analyses, plans, implements and technically tests IT solutions in the context of projects within the scope of the DataWarehouse. Ensures the formalisation and ensures that documentation is completed in accordance with current procedures. Occasionally participates in meetings with the Business Areas and in working sessions with the team responsible for functional definition, carrying out certification/quality tests and monitoring UAT tests. Proposes improvements to management platforms and tools, with the aim of optimising and innovating them. Formalises and documents the technical analyses it carries out, the technical solutions it implements, the technical tests it performs and the decision‐making meetings it participates in. Communicates and validates its activities and decisions with the person in charge, thus ensuring that they are aligned with the department's recommendations in terms of posture and Roadmap. Reports to the Data Hub Chapter.

What are we looking for?

• Experience in the environment of the service : >6 years
• Ensure implementation of data access, referencing, and quality
• Responsible for implementing raw data ingestion pipeline (Data Lake or other)
• SQL / NO SQL expertise
• Ensure industrialisation, supervision, and integration of solutions or data while guaranteeing the service commitment of the production environment, etc.
• Expert in Hadoop zoo Ecosystem
• Knowledge / Mastery of Data
• Knowledge / Technical mastery of the DevOps environment
• Technical mastery of the Data environment
• Expert in Access and Exchange layer (Kafka, Nifi, etc.)
• Expert in Java, Scala, Python, Hive, Spark, etc.
• Expert in ETL and Bash
• Power Bi 3
• Jira/Confluence/ Draw IO
• Knowledge of applications from a user perspective: MS Outlook, Word, Excel and PowerPoint
• Bachelor's or Master's degree in Computer Engineering, Information Technology or similar areas
• Experience in Datawarehouse solutions, with an understanding of all phases of construction
• Minimum of 4 years of experience in functional/technical analysis and definition of corporate data models
• Experience in projects/environments with Agile framework
• Preference for experience in the Banking area (consumer credit)
• Strong communication and organizational skills
• Ability to gather requirements, document and lead meetings
• Autonomy and proactivity. Critical thinking
• Ability to synthesize and solve problems. Adaptability and resilience
• Focus on the customer and delivering results

What can you expect from us?

• A permanent job contract for a long term project;
• Tech equipment + SIM Card + personal smartphone;
• Health and Life Insurance;
• Social events and team buildings;
• The commitment of letting you grow with us, and be rewarded accordingly;
• A dynamic and young team that will be always there to support you;
• Training in the latest technologies;
• Coffee, fruits, snacks and a warm welcoming when you pass by the office.