Big Data Engineer

Aubay Portugal; Viseu; Presencial; EUR 60 000 - 90 000; Tempo integral

Resumo da oferta

Inetum seeks a seasoned Big Data Engineer to join our team, supporting Hadoop-based data lake environments. The role focuses on performance, reliability, and security of the big data infrastructure, with responsibilities spanning operations, troubleshooting, and resource management.

You will work across cross-functional teams to onboard workloads, implement patches, and maintain Kerberos, Ranger/Sentry policies, and TLS encryption, while documenting cluster configurations and sharing insights.

Qualificações

• Bachelor’s degree in Computer Science or equivalent.
• Minimum 5 years of experience in a similar role.
• Strong expertise in Hadoop Ecosystem & Data Lake Operations.
• Strong expertise in Big Data Services & Query Engines.
• Security & Access Control in Distributed Environments.
• Monitoring, Automation & Platform Lifecycle Management.
• Proven ability to diagnose and resolve complex issues in Big Data environments.
• Strong communication skills to collaborate with cross-functional teams.
• English proficiency: B2-C1 level.

Responsabilidades

• Operate, monitor, and support Hadoop-based data lake platforms.
• Provide L3 incident response, deep troubleshooting, and performance tuning for big data components.
• Ensure data integrity, replication, and capacity management within HDFS clusters.
• Develop and automate monitoring and alerting for service health and node performance.
• Collaborate with platform and data teams to onboard new workloads with optimal resource allocation.
• Apply patches, upgrades, and configuration changes to maintain platform security and stability.
• Manage Kerberos authentication, Ranger/Sentry policies, and TLS encryption.
• Work with storage, network, and security teams to optimize throughput and access controls.
• Maintain cluster documentation and share knowledge across teams.

Conhecimentos

Hadoop Ecosystem & Data Lake Ops

Big Data Services & Query Engines

Security & Access Control in Dist. Env

Monitoring, Automation & Lifecycle Mgm

English Proficiency (B2-C1)

Formação académica

Bachelor’s degree in Computer Science or equivalent

Descrição da oferta de emprego

Descrição do cargo

Job Description

Inetum is seeking a seasoned Big Data Engineer to join our team and support the operations of our Hadoop-based data lake environments. This is a critical role for ensuring the performance, reliability, and security of our big data infrastructure.

Key Responsibilities

• Operate, monitor, and support Hadoop-based data lake platforms.
• Provide L3 incident response, deep troubleshooting, and performance tuning for big data components.
• Ensure data integrity, replication, and capacity management within HDFS clusters.
• Develop and automate monitoring and alerting for service health and node performance.
• Collaborate with platform and data teams to onboard new workloads with optimal resource allocation.
• Apply patches, upgrades, and configuration changes to maintain platform security and stability.
• Manage Kerberos authentication, Ranger/Sentry policies, and TLS encryption.
• Work with storage, network, and security teams to optimize throughput and access controls.
• Maintain cluster documentation and share knowledge across teams.

Profile

• Bachelor’s degree in Computer Science or equivalent.
• Minimum 5 years of experience in a similar role.
• Strong expertise in:
  • Hadoop Ecosystem & Data Lake Operations
  • Big Data Services & Query Engines
  • Security & Access Control in Distributed Environments
  • Monitoring, Automation & Platform Lifecycle Management
• Proven ability to diagnose and resolve complex issues in Big Data environments.
• Strong communication skills to collaborate with cross-functional teams.
• English proficiency: B2-C1 level

About Inetum

Inetum is a European leader in digital services. Inetum’s team of 28,000 consultants and specialists strive every day to make a digital impact for businesses, public sector entities and society. Inetum’s solutions aim at contributing to its clients’ performance and innovation as well as the common good.

Present in 19 countries with a dense network of sites, Inetum partners with major software publishers to meet the challenges of digital transformation with proximity and flexibility.

Driven by its ambition for growth and scale, Inetum generated sales of 2.5 billion euros in 2023.

Additional Information

Nice to Have:

• Big Data certifications (e.G., Cloudera).
• Familiarity with Kafka, NiFi, or Flume for ingestion pipelines.
• Knowledge of ITIL processes for incident and change management.