Walk-in | Lead Data Engineer

The Ksquare Group; Hyderabad; Hybrid; INR 4.000.000 - 7.000.000; Vollzeit

Zusammenfassung

The Ksquare Group seeks a Tech Lead Data Engineering to own the technical delivery of enterprise data platforms powering KPI reporting and analytics. You will collaborate with data architects, DataOps, application teams, and stakeholders to build scalable pipelines and analytic models.

Responsibilities include data profiling, de-duplication, standardization, governance, and KPI instrumentation. You will mentor engineers and lead a small team toward an AI-accelerated timeline.

Qualifikationen

• 8+ years in data engineering with 3+ years leading technical teams on large-scale cleansing/migration, data reconciliation and data quality.
• Hands-on data engineering experience building production ETL/ELT pipelines.
• Proficient in Python, Pyspark, PostgreSQL, SQL.
• Experience with GCP Cloud (BigQuery and related tech).
• Data warehousing, data modelling, data normalization.
• Hands-on de-duplication and entity resolution (probabilistic/fuzzy matching, standardization).
• Orchestration tools like Airflow/Dataflow.

Aufgaben

• Lead full AI-assisted profiling of data to size duplicates, placeholders, and stale records ahead of cleansing.
• Design and execute probabilistic de-duplication and name/address standardization across legacy datasets.
• Implement unique-identifier anchoring across records for a trusted registry.
• Stand up a data governance framework with stewardship roles and data-quality KPIs.
• Define and instrument data quality KPIs/SLAs with governance dashboards.
• Ensure API and data model support for offline-first synchronization for low-connectivity regions.
• Set engineering standards, review designs, and lead a data engineers/analysts team.
• Act as the primary technical contact for discovery, profiling, and dependency lock-down.
• Escalate risks around dependencies and SME availability with proactive communication.
• Support UAT, training, and governance-rule sustainment post go-live.
• Mentor junior and mid-level data engineers.

Kenntnisse

Python

Pyspark

PostgreSQL

SQL

GCP Cloud

BigQuery

Cloud Storage

Cloudrun

Dataflow

Dataform

Airflow

Hadoop

Hive

Spark

Tools

Airflow

Dataflow

Jobbeschreibung

We're Hiring: Tech Lead Data Engineering

Are you passionate about building scalable data platforms and driving data quality initiatives? We are looking for a Tech Lead - Data Engineering to lead the technical delivery of enterprise data platforms that power business KPI reporting and analytics.

About the Role

We are seeking a **Tech Lead Data Engineering** to own the technical delivery of Data platforms which will be used to for Business KPI reporting.The resource will work closely with data architects, DataOps engineers, application teams, and business stakeholders to build scalable, secure, and high-performing data pipelines and analytical data models .This role requires someone proficient in Data Modelling/Profiling/Cleansing/de-duplication/ and Data Reconciliation as per the business rules.

Key Responsibilities

• Lead full AI-assisted profiling of the data to precisely size duplicates, placeholder entries, and stale/conflicting records ahead of cleansing.
• Design and execute **probabilistic de-duplication** and name/address standardization across legacy and data.
• Implement unique-identifier anchoring across every record so all programs that operate from one trusted registry.
• Stand up the **ongoing data governance framework** that protects the cleansed registry going forward: stewardship roles, validation-at-entry rules, controlled reference data, change control, and data-quality KPIs.
• Define and instrument data quality KPIs/SLAs (completeness, duplication rate, freshness, identifier-match confidence) with dashboards visible to program governance and leadership.
• Ensure the API and underlying data model support conflict-safe synchronization for the offline-first intake application (store-and-forward sync from low-connectivity regions).
• Set engineering standards, review technical designs, and lead a small team of data engineers/analysts through an aggressive, AI-accelerated timeline.
• Act as the primary technical point of contact for the Deep-Dive Discovery: validating current state, running a data-profiling preview to size the cleansing effort, and locking down dependencies.
• Escalate and manage risk around known dependencies schedule slippage, SME availability so proactive tracking and communication is essential.
• Support UAT, training, and hyper care for data-quality-related workflows; ensure data stewards are equipped to sustain governance rules post-go-live.
• Mentor Junior and mid-level data engineers

Mandatory Skills

• 8+ years in data engineering, with 3+ years of experience leading technical teams on large-scale Data cleansing/migration,Data reconciliation and Data quality.
• Hands-on data engineering experience building production ETL/ELT pipelines.
• Proficient in Python,Pyspark,PostgreSQL,SQL
• Experience with GCP Cloud(BigQuery and related tech like Cloud Storage, Cloudrun, Dataflow and dataform)
• Datawarehousing,Data Modelling,Data Normalization.
• Demonstrated, hands-on experience with large-scale data de-duplication and entity resolution (probabilistic/fuzzy matching, standardization of names/addresses, unique-identifier anchoring)
• Orchestration tools like Airflow/Dataflow

Nice to Have Skills

• Data governance and Data Lineage
• Good Understanding of API's design(RESTful/secure API layers over a system of record, versioning, documentation) suitable for multiple consumer channels (web portal, offline/mobile, future integrations).
• Understanding of PII/data protection,data masking
• Excellent stakeholder communication skills - ability to present progress, risk, and data quality metrics to leadership and program governance.
• Experience with other clouds like AWS,Azure and modern datawarehouses like Databricks/Snowflake.
• CI/CD, github
• Big Data technologies Hadoop, HDFS, Hive, Apache Spark