Data Platform Engineer

Maimsd Technology Private Limited; India; Remote; INR 1,500,000 - 2,500,000; Full time

Job summary

Maimsd Technology Private Limited invites a Data Platform Engineer with hands-on experience in HBase, Amazon EMR, Apache Spark, and Scala to join our remote team. You will manage production data platforms, migrate and modernize HBase workloads, and ensure data integrity during large-scale migrations.

Responsibilities include handling HBase snapshots, HFiles, and AWS-based data platforms, coordinating with Cloud, Architecture, Backend, DevSecOps, Security, and QA teams, and supporting migrations

Qualifications

• 7+ years of IT experience with 6+ years in data engineering or big data.
• Minimum 5 years of Data/Big Data Engineering experience.
• Hands-on production experience with Apache HBase.
• Production experience with Amazon EMR.
• Hands-on Apache Spark and Scala experience.
• Strong understanding of the Hadoop ecosystem.
• Experience with AWS-based data platforms and S3.
• Experience in HBase migration and modernization.

Responsibilities

• Design, manage, and troubleshoot production HBase and distributed data platforms.
• Execute HBase migration and modernization across clusters.
• Work with HBase snapshots, Restore, ExportSnapshot, and large-scale data movement.
• Manage Amazon EMR clusters, including config and upgrades.
• Develop and troubleshoot Spark applications using Scala.
• Plan production migrations, cutovers, rollback, and post-migration validation.
• Ensure data integrity and validation during migrations.

Skills

HBase

Amazon EMR

Apache Spark

Scala

Hadoop ecosystem

AWS

Data migration

Production troubleshooting

Tools

HBase snapshots

ExportSnapshot

HFiles

Restore procedures

Amazon S3

Job description

Job Title: Data Platform Engineer – HBase / Amazon EMR / Spark

Number of Openings: 5

Experience: 7+ Years

Relevant Experience: 6+ Years

Employment Type: Full-Time

Work Mode: Remote

Shift Timing: 3:00 PM – 12:00 AM IST

Notice Period: Immediate Joiners or candidates serving notice up to 20 days

Job Summary

We are looking for experienced Data Platform Engineers with strong hands-on expertise in HBase, Amazon EMR, Apache Spark, Scala, and the Hadoop ecosystem. The ideal candidate will have proven experience managing production data platforms, migrating and modernizing HBase workloads, troubleshooting distributed systems, and ensuring data integrity during large-scale production migrations.

The role requires practical experience in HBase snapshots, HFiles, data movement, migration planning, production cutover, rollback procedures, and post-migration validation within AWS-based data platforms.

Key Responsibilities

• Design, manage, maintain, and troubleshoot production HBase and distributed data platforms.
• Execute HBase migration and modernization activities across clusters and environments.
• Work with HBase snapshots, Restore, ExportSnapshot, HFiles, and large-scale data movement techniques.
• Manage Amazon EMR clusters, including configuration, troubleshooting, upgrades, and workload migration.
• Develop, optimize, and troubleshoot Apache Spark applications using Scala and/or other relevant Spark programming.
• Work with Hadoop ecosystem components and understand dependencies between HBase, Spark, Hadoop, and application workloads.
• Integrate and manage data platforms using Amazon S3 and other AWS services.
• Plan and execute production migrations, cutovers, rollback strategies, and post-migration support.
• Validate migrated data using row counts, region counts, reconciliation, checksums, and other data integrity checks.
• Troubleshoot cluster-level issues related to performance, availability, connectivity, configuration, and compatibility.
• Ensure application stability and data consistency throughout migration and modernization activities.
• Develop automation s, operational runbooks, and repeatable migration and validation processes.
• Collaborate with Cloud, Architecture, Backend, DevSecOps, Security, and QA teams to ensure successful migration delivery.

Mandatory Skills and Experience

• 7+ years of overall IT experience, including 6+ years of relevant experience.
• Minimum 5 years of experience in Data Engineering or Big Data Engineering.
• Strong hands-on production experience with Apache HBase.
• Strong production experience with Amazon EMR.
• Hands-on experience with Apache Spark and Scala and/or Spark programming.
• Strong understanding of the Hadoop ecosystem and distributed data platforms.
• Experience working with AWS-based data platforms and Amazon S3.
• Proven experience in HBase migration, data platform modernization, or cluster migration.
• Practical knowledge of HBase snapshots, ExportSnapshot, HFiles, and restore procedures.
• Experience executing production data migration, cutover, rollback, and post-migration validation.
• Ability to validate data integrity using counts, reconciliation, checksums, or equivalent techniques.
• Strong troubleshooting skills involving distributed clusters and production workloads.
• Understanding of dependencies between HBase, Spark, Hadoop, and business applications.
• Experience in production support and migration-related issue resolution.

Good-to-Have Skills

• Experience with HBase 1.x/2.x and Amazon EMR 5.x/7.x.
• Knowledge of HBase Coprocessors and Apache Phoenix.
• Exposure to Hive, Presto, Apache Iceberg, Kerberos, and AWS KMS.
• Experience with encrypted data platforms and enterprise security requirements.
• Experience with large-scale enterprise data platform modernization.
• Strong documentation, automation, and cross-functional collaboration skills.

Candidate Profile

The ideal candidate should have strong analytical and problem-solving skills, hands-on production troubleshooting experience, and the ability to independently execute complex data migration activities. Candidates should be comfortable working in a remote environment during the specified shift.

Important Requirements

• Immediate joiners or candidates with a notice period of up to 20 days only.
• Must be comfortable working remotely from 3:00 PM to 12:00 AM IST.
• Candidates must have demonstrable hands-on experience in HBase, Amazon EMR, and HBase migration/modernization.