Mid-Level Hadoop Data Lake Engineer

DataSmart; Lisboa; Hybrid; EUR 45,000 - 70,000; Full time

Job summary

DataSmart in Portugal is seeking an experienced Data Platform Engineer to operate and support Hadoop-based data lake platforms. You will monitor performance, ensure data integrity, and collaborate with cross-functional teams to onboard new workloads.

The role emphasizes L3 incident management, security via Kerberos and TLS, and applying patches to keep clusters stable while maintaining clear documentation and knowledge sharing across teams. Join DataSmart for national and international projects.

Qualifications

• Extensive experience operating Hadoop-based data platforms.
• Experience in L3 support and incident management.
• Strong data integrity, security, and monitoring capabilities.

Responsibilities

• Operate, monitor, and support Hadoop-based data lake platforms.
• Provide L3 incident response and deep troubleshooting.
• Ensure data integrity, replication, and capacity management in HDFS.
• Develop and automate monitoring and alerting for services and node health.
• Collaborate with platform and data teams to onboard workloads.
• Apply patches, upgrades, and config changes to maintain stability.
• Manage authentication and security using Kerberos.
• Implement access control with Ranger/Sentry.
• Collaborate with storage, network, and security teams to optimize performance.
• Maintain cluster documentation and share knowledge across teams.

Skills

Hadoop ops

L3 support

Performance tuning

Data replication

Security & authentication

Kerberos

TLS

Monitoring

Cross-functional teams

Tools

Apache Ranger

Apache Sentry

HDFS

Job description

DataSmart in Portugal is seeking an experienced Data Platform Engineer to operate and support Hadoop-based data lake platforms. You will monitor performance, ensure data integrity, and collaborate with cross-functional teams to onboard new workloads.

The role emphasizes L3 incident management, security via Kerberos and TLS, and applying patches to keep clusters stable while maintaining clear documentation and knowledge sharing across teams. Join DataSmart for national and international projects.