Iceberg DBA / Lakehouse Operations Engineer_remote in dallas,TX

Floga Technologies; Dallas (TX); Hybrid; USD 140,000 - 190,000; Full time

Job summary

Floga Technologies is seeking an Iceberg DBA / Lakehouse Operations Engineer to manage Apache Iceberg data layers supporting enterprise analytics. The role covers data reliability, multi-engine query performance across Spark/Spark SQL, Hive, and Impala, and modernization efforts from Hive/Teradata to Iceberg.

Remote-friendly with flexibility for Dallas, TX, based work. The candidate will oversee table maintenance, metadata management, partitioning strategies, and data modeling within a Medallion

Qualifications

• Strong hands-on experience with Apache Iceberg and/or Hive-based data lake environments.
• Strong understanding of Iceberg table architecture, metadata management, and table maintenance operations.
• Experience with table-level optimization, compaction, snapshot management, and performance tuning.
• Understanding of data modeling concepts, including normalization and 3NF.
• Knowledge of modern Lakehouse architecture and Medallion architecture (Bronze/Silver/Gold).
• Hands-on experience with Spark SQL, Hive, and Impala.
• Working knowledge of NiFi and Trino.
• Strong understanding of partitioning strategies, partition evolution, and partition pruning.
• Experience with columnar file formats such as Parquet and ORC.
• Understanding of distributed query processing and large-scale data management.
• Experience supporting multi-terabyte to petabyte-scale data platforms.

Responsibilities

• Iceberg Data Layer Ownership & Operations
• Own day-to-day operations and administration of Apache Iceberg tables supporting multiple enterprise applications.
• Ensure data reliability, consistency, and availability across Lakehouse workloads.
• Maintain operational integrity of datasets at multi-terabyte to petabyte scale.
• Monitor table health, metadata growth, and storage utilization.
• Ensure consistent data access and operational stability across environments.
• Advanced Table Management & Optimization: Minor and major compaction; Small file mitigation and optimization; Snapshot expiration and metadata compaction; Orphan file cleanup and storage management.
• Optimize data layout and query performance through file sizing, distribution, and partitioning strategies.
• Implement partition evolution and partition pruning techniques.
• Apply clustering and data ordering strategies, such as Z-ordering or equivalent approaches.
• Monitor and optimize table maintenance operations to improve query performance and storage efficiency.
• Data Modeling & Lakehouse Architecture: 3NF; Medallion architecture; align with ingestion patterns.
• Balance data integrity, normalization, and analytical query performance.
• Collaborate with data engineering teams to ensure consistent implementation of layered data architecture across applications.
• Multi-Engine Query Performance & Consistency: Spark SQL, Hive, Impala; troubleshoot bottlenecks and RCAs across engines.
• Hive & Teradata Modernization Support: migrations to Iceberg; schema alignment; data validation and reconciliation.
• Metadata & Data Lifecycle Management: manage Iceberg metadata, retention, snapshot lifecycle; governance and access controls.
• Production Support & Incident Management: support for Iceberg Lakehouse workloads; on-call readiness; RCA and corrective actions.
• Security & Data Governance: RBAC and Apache Ranger policies.

Skills

Apache Iceberg

Hive data lake

Iceberg metadata management

Table maintenance optimization

Data modeling 3NF

Lakehouse architecture

Spark SQL / Hive / Impala

NiFi

Trino

Partitioning strategies

Parquet / ORC formats

Large-scale data platforms

Tools

NiFi

Trino

Job description

Good morning, I'm

Hello, Good morning, I'm Rahul from Floga Technologies. We are a growing IT services organization delivering scalable technology solutions and are expanding our team with qualified talent.

We have an immediate opening for a Iceberg DBA / Lakehouse Operations Engineer Experience with strong hands-on experience to support a long-term engagement with our client. Must collaborate effectively with cross-functional teams to deliver high-quality solutions.

Job Title: Iceberg DBA / Lakehouse Operations Engineer Location: Remote (Based on the business needs candidates needs to be flexible to work from office, Dallas, TX) Experience: 10+ years Duration: Long Term

Job Summary

We are seeking a highly skilled Iceberg DBA / Lakehouse Operations Engineer to manage the reliability, performance, and operational integrity of the Apache Iceberg data layer supporting enterprise analytics and business-critical applications.

The role involves managing a large-scale, multi-engine Lakehouse environment supporting workloads across Spark, Hive, and Impala. The engineer will play a key role in enterprise data modernization initiatives, particularly migrations from Hive and Teradata to Apache Iceberg.

The ideal candidate will have strong hands-on experience with Iceberg table operations, metadata management, data modeling, and query performance optimization in cloud-based environments.

This role is critical to maintaining data accuracy, availability, and performance, as any degradation can directly impact downstream reporting, analytics, and business-critical decision-making.

Key Responsibilities

• Iceberg Data Layer Ownership & Operations
• Own day-to-day operations and administration of Apache Iceberg tables supporting multiple enterprise applications.
• Ensure data reliability, consistency, and availability across Lakehouse workloads.
• Maintain operational integrity of datasets at multi-terabyte to petabyte scale.
• Monitor table health, metadata growth, and storage utilization.
• Ensure consistent data access and operational stability across environments.
• Advanced Table Management & Optimization
  • Perform advanced Iceberg table maintenance and optimization activities, including:
  • Minor and major compaction.
  • Small file mitigation and optimization.
  • Snapshot expiration and metadata compaction.
  • Orphan file cleanup and storage management.
• Optimize data layout and query performance through file sizing, distribution, and partitioning strategies.
• Implement partition evolution and partition pruning techniques.
• Apply clustering and data ordering strategies, such as Z-ordering or equivalent approaches.
• Monitor and optimize table maintenance operations to improve query performance and storage efficiency.
• Data Modeling & Lakehouse Architecture
  • Support and enforce data modeling best practices, including:
  • Third Normal Form (3NF) for source-aligned datasets.
  • Medallion architecture (Bronze, Silver, and Gold) for curated data pipelines.
• Ensure Iceberg table designs align with ingestion patterns, downstream consumption, and performance requirements.
• Balance data integrity, normalization, and analytical query performance.
• Collaborate with data engineering teams to ensure consistent implementation of layered data architecture across applications.
• Multi-Engine Query Performance & Consistency
  • Ensure consistent and optimized query performance across:
  • Apache Spark / Spark SQL (CDE).
  • Hive.
  • Impala (CDW).
• Troubleshoot query performance bottlenecks, metadata inconsistencies, and inefficient execution plans.
• Analyze query execution plans, scan patterns, and data access behavior.
• Ensure consistent table metadata and query results across multiple processing engines.
• Hive & Teradata Modernization Support
  • Support enterprise data modernization initiatives involving Hive-to-Iceberg and Teradata-to-Iceberg migrations.
  • Assist with schema alignment, data type mapping, and table structure conversion.
  • Perform data validation, reconciliation, and accuracy checks between source and target datasets.
  • Troubleshoot migration-related issues and ensure post-migration stability and performance.
  • Collaborate with data engineering teams to optimize migrated workloads.
• Metadata & Data Lifecycle Management
  • Manage Iceberg metadata to ensure scalability, consistency, and performance.
  • Maintain consistent table states and metadata across multiple query engines.
  • Implement data retention and archival policies.
  • Manage snapshot lifecycle operations, expiration, and cleanup.
  • Optimize time-travel capabilities and metadata maintenance.
  • Monitor metadata growth and prevent performance degradation.
• Production Support & Incident Management
  • Provide L2/L3 operational support for production issues across Iceberg-based Lakehouse workloads.
  • Participate in on-call rotations to support critical data platforms.
  • Respond to and resolve P1/P2 incidents within defined SLAs.
  • Troubleshoot data inconsistencies, reporting discrepancies, query failures, and performance degradation.
  • Perform Root Cause Analysis (RCA) and implement permanent corrective and preventive actions.
  • Collaborate with platform, infrastructure, and application teams during incident triage and resolution.
• Security & Data Governance
  • Support fine-grained data access controls using Apache Ranger policies and Role-Based Access Control (RBAC).
  • Ensure data validation, reconciliation, and accuracy between source systems and Iceberg datasets.
  • Maintain secure and compliant data access across applications and processing environments.
  • Collaborate with security and governance teams to enforce enterprise data access policies.

Required Skills & Qualifications

• Strong hands-on experience with Apache Iceberg and/or Hive-based data lake environments.
• Strong understanding of Iceberg table architecture, metadata management, and table maintenance operations.
• Experience with table-level optimization, compaction, snapshot management, and performance tuning.
• Understanding of data modeling concepts, including normalization and 3NF.
• Knowledge of modern Lakehouse architecture and Medallion architecture (Bronze/Silver/Gold).
• Hands-on experience with Spark SQL, Hive, and Impala.
• Working knowledge of NiFi and Trino.
• Strong understanding of partitioning strategies, partition evolution, and partition pruning.
• Experience with columnar file formats such as Parquet and ORC.
• Understanding of distributed query processing and large-scale data management.
• Experience supporting multi-terabyte to petabyte-scale data platforms.

Preferred Skills

• Experience with Hive-to-Iceberg and/or Teradata-to-Iceberg migrations.
• Experience with Cloudera CDP, particularly CDE and CDW.
• Familiarity with cloud platforms such as AWS and Azure.
• Experience with Apache Ranger, data governance, and access management.
• Scripting and automation experience using Python and Shell.
• Experience supporting enterprise-scale Lakehouse modernization initiatives.

Thanks & Regards,

Rahul H | Floga Technologies.

Sr. Talent Acquisition Specialist E: rahul