Data Architect – Big Data/Open Source

Edgematics Technologies LLC; Pune District; Hybrid; INR 2,000,000 - 3,500,000; Full time

Job summary

Edgematics Technologies LLC in Pune is looking for an experienced Enterprise Data Architect (Lakehouse & Governance) to lead architecture design and implementation of enterprise-scale Telecom Datalake. This hands-on role centers around governance and efficient data management. The ideal candidate will have at least 12–15 years of experience and expertise in Spark, Iceberg, and related technologies.

The position offers a hybrid work model and requires strong leadership and communication skills.

Qualifications

• 12–15 years of experience in Big Data and distributed systems architecture.
• Strong hands-on expertise with Spark, Iceberg, Trino, NiFi, dbt, and object storage platforms.
• Proven experience designing enterprise-scale Datalake and Datamart architectures.

Responsibilities

• Define and design scalable end-to-end Big Data architecture.
• Architect Iceberg-based Lakehouse solutions integrated with Spark and Trino.
• Design and implement enterprise Data Governance frameworks.

Skills

Spark

Iceberg

Trino

NiFi

dbt

Kubernetes

Data Governance solutions

Tools

MinIO

S3

Job description

Job Details

• Experience: 12–15 Years
• Location: Pune / Hybrid
• Company: Edgematics Group

Job Overview

We are seeking an experienced Enterprise Data Architect (Lakehouse & Governance) to lead the end-to-end architecture design and implementation of enterprise-scale Telecom Datalake, Datamart, and Data Governance platforms. This is a hands‑on, governance‑heavy leadership role aligned with large‑scale delivery expectations. The architect will define scalable, secure, and high‑performance Lakehouse architectures while ensuring tight coordination across Platform Engineering, Governance, BI, and DevOps streams. In alignment with the proposed stack, the architecture focus includes NiFi, dbt, CI/CD orchestration, Iceberg, Spark, and Trino, avoiding dependency on non‑proposed orchestration tools.

Key Responsibilities

• Define and design scalable end‑to‑end Big Data architecture across ingestion, storage, processing, governance, and consumption layers.
• Architect Iceberg‑based Lakehouse solutions integrated with Spark, Trino, NiFi, dbt, and object storage (MinIO/S3).
• Define orchestration strategy using dbt + CI/CD + platform‑native orchestration (where applicable).
• Design and implement enterprise Data Governance frameworks using Atlas and Ranger.
• Establish high availability (HA), disaster recovery, workload isolation, and performance optimization strategies.
• Define best practices for metadata management, fine‑grained access control, and data lifecycle management.
• Lead performance benchmarking ownership, capacity planning, and scalability validation for Telecom‑scale workloads.
• Architect Telecom domain data models (Customer, Usage, Network, Billing domains) aligned with Datamart consumption.
• Design and govern Data Migration Factory frameworks for legacy‑to‑Lakehouse transitions.
• Provide architectural governance, delivery oversight, and mentoring across Platform Engineer, Governance Engineer, BI Engineer, and DevOps Engineer roles.
• Evaluate emerging technologies and recommend modernization strategies aligned with enterprise and regulatory objectives.

Requirements

• 12–15 years of experience in Big Data and distributed systems architecture.
• Strong hands‑on expertise with Spark, Iceberg, Trino, NiFi, dbt, and object storage platforms (MinIO/S3).
• Proven experience designing enterprise‑scale Datalake and Datamart architectures.
• Mandatory OpenShift / Kubernetes production architecture experience.
• Strong experience in Telecom data models (Customer, Usage, Network, Billing domains).
• Experience designing and governing Data Migration Factory implementations.
• Proven ownership of performance benchmarking and platform scalability validation.
• Deep understanding of distributed systems, concurrency planning, workload isolation, and query optimization.
• Experience implementing enterprise Data Governance solutions (Atlas, Ranger or similar tools).
• Strong knowledge of hybrid or cloud‑native infrastructure environments.
• Excellent stakeholder management, executive communication, and leadership skills.