data engineer in big data infrastructure

HireHi; Lisboa; Presencial; EUR 60 000 - 90 000; Tempo integral

Resumo da oferta

Inetum in Lisbon seeks an experienced Big Data Engineer to operate and optimize Hadoop-based data lake platforms, ensuring security and high availability. You will handle L3 incident response, performance tuning, and collaborate with cross-functional teams to onboard new workloads while maintaining data integrity and robust access controls.

Minimum 5 years in a similar role with strong Hadoop expertise, data lake operations, and familiarity with Kafka, NiFi, or Flume.

Qualificações

• Bachelor’s degree in Computer Science or equivalent.
• Minimum 5 years of experience in a similar role.
• Strong expertise in the Hadoop ecosystem and data lake operations.
• Strong expertise in big data services and query engines.
• Strong expertise in security and access control in distributed environments.
• Strong expertise in monitoring, automation, and platform lifecycle management.
• Proven ability to diagnose and resolve complex issues in big data environments.
• Strong communication skills for collaboration with cross-functional teams.
• English proficiency at B2-C1 level.
• Nice to have: Cloudera, Kafka, NiFi, Flume; ITIL knowledge.

Responsabilidades

• Operate, monitor, and support Hadoop-based data lake platforms.
• Provide L3 incident response, deep troubleshooting, and performance tuning for big data components.
• Ensure data integrity, replication, and capacity management within HDFS clusters.
• Develop and automate monitoring and alerting for service health and node performance.
• Collaborate with platform and data teams to onboard new workloads with optimal resource allocation.
• Apply patches, upgrades, and configuration changes to maintain platform security and stability.
• Manage Kerberos authentication, Ranger/Sentry policies, and TLS encryption.
• Work with storage, network, and security teams to optimize throughput and access controls.
• Maintain cluster documentation and share knowledge across teams.

Conhecimentos

Hadoop ecosystem

Data lake ops

Big data services

Security access control

Monitoring automation

Platform lifecycle

Cross-functional comms

English B2-C1

Kafka

NiFi

Flume

Cloudera

Formação académica

Bachelor's degree in Computer Science or equivalent

Ferramentas

Kafka

NiFi

Flume

Cloudera

Descrição da oferta de emprego

Описание:

Inetum is a European leader in digital services, providing solutions that support business and public-sector performance, innovation, digital transformation, and the common good.

Задачи:

• Operate, monitor, and support Hadoop-based data lake platforms;


• Provide L3 incident response, deep troubleshooting, and performance tuning for big data components;


• Ensure data integrity, replication, and capacity management within HDFS clusters;


• Develop and automate monitoring and alerting for service health and node performance;


• Collaborate with platform and data teams to onboard new workloads with optimal resource allocation;


• Apply patches, upgrades, and configuration changes to maintain platform security and stability;


• Manage Kerberos authentication, Ranger/Sentry policies, and TLS encryption;


• Work with storage, network, and security teams to optimize throughput and access controls;


• Maintain cluster documentation and share knowledge across teams.

Требования:

• Bachelor's degree in Computer Science or equivalent;


• Minimum 5 years of experience in a similar role;


• Strong expertise in the Hadoop ecosystem and data lake operations;


• Strong expertise in big data services and query engines;


• Strong expertise in security and access control in distributed environments;


• Strong expertise in monitoring, automation, and platform lifecycle management;


• Proven ability to diagnose and resolve complex issues in big data environments;


• Strong communication skills for collaboration with cross-functional teams;


• English proficiency at B2-C1 level;


• Nice to have: Big Data certifications such as Cloudera, familiarity with Kafka, NiFi, or Flume for ingestion pipelines, knowledge of ITIL processes for incident and change management.

Условия:

No conditions specified