Data Engineering Technical Lead

Texplorers Inc; Hyderabad; On-site; INR 3,500,000 - 7,500,000; Full time

Job summary

Texplorers Inc. seeks a Technical Lead to drive migration of Microsoft-based data platforms to Databricks, providing hands-on mentorship and architectural oversight for the data engineering team.

The role is pure technical leadership, focused on architecture and delivery rather than SCRUM facilitation. The position emphasizes technical direction, delivery execution, removal of blockers, and ensuring scalable, consistent solutions across teams.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, or a strictly related technical discipline.
• 7-10 years of specialized experience in Data Engineering.
• Preferred background in the Retail sector, with an understanding of industry-specific data challenges.

Responsibilities

• Act as the technical reference point for data engineers during migration and transformation activities.
• Break down high-level migration objectives into clear technical tasks and implementation approaches.
• Lead the migration of data pipelines from Azure SQL / SQL Managed Instances into Databricks (Lakehouse architecture).
• Define and enforce best practices for transformation patterns and performance optimization.
• Contribute directly to development when required (Databricks notebooks, PySpark/SQL).
• Mentor data engineers and help upskill them towards Databricks and modern data engineering practices.
• Facilitate technical discussions, refinement sessions, and solution reviews.
• Ensure development stays on track across multiple parallel workstreams.

Skills

Databricks

PySpark

Spark SQL

Lakehouse

Azure Cloud

SQL Server

Data Engineering

Leadership

Education

Bachelor’s or Master’s degree in Computer Science or related technical discipline

Tools

Git

Job description

The Technical Lead will drive the migration of our Microsoft-based data platforms (on-premise and Azure) to Databricks, providing hands‑on mentorship and technical oversight for the data engineering team.

Please note: This is a pure technical leadership role, focused on architecture and execution rather than project management or Scrum facilitation.

The focus is on:

Technical direction

Delivery execution

Removing technical blockers

Ensuring consistency and scalability across teams

The Technical Lead will act as the bridge between architecture, engineering teams, and delivery, while remaining capable of hands‑on contribution when needed.

Key Responsibilities

Technical Leadership & Coordination

Act as the technical reference point for data engineers during migration and transformation activities

Break down high‑level migration objectives into clear technical tasks and implementation approaches

Ensure alignment to target architecture and standards (Databricks, cloud‑native patterns)

Review designs, pipelines, and implementations to ensure quality and consistency

Actively remove technical blockers for the engineering teams

Migration & Transformation Execution

Lead the migration of data pipelines from:

oAzure SQL / SQL Managed Instances into Databricks (Lakehouse architecture)

Define and enforce best practices for:

oTransformation patterns

oPerformance optimization

Support coexistence scenarios during the transition phase

Hands-on Engineering (as needed)

Contribute directly to development when required:

Databricks notebooks (PySpark / SQL)

Ingestion pipelines

Refactoring legacy logic

Build reference implementations and examples for the team

Support troubleshooting and performance tuning

Team Enablement

Mentor data engineers and help upskill them towards Databricks and modern data engineering practices

Facilitate technical discussions, refinement sessions, and solution reviews

Ensure development stays on track, even across multiple parallel workstreams

What This Role Is NOT

Not a Project Manager

Not a Scrum Master

Not a pure Architect

It is a delivery‑focused technical leadership role.

Required Technical Skills

Core Platform Knowledge

Strong experience in Data Engineering

Proven experience leading engineers in complex data environments

Hands‑on experience with:

Databricks (mandatory)

PySpark & Spark SQL

Lakehouse concepts (Bronze / Silver / Gold)

Microsoft & Azure Background (must‑have due to migration context)

Solid experience with:

SQL Server

Ability to translate legacy ETL logic into modern Spark‑based pipelines

Experience with Azure cloud environments

Understanding of:

Git‑based development

Infrastructure‑as‑code concepts (nice to have)

Performance tuning and troubleshooting experience

Qualifications

Bachelor’s or Master’s degree in Computer Science, or a strictly related technical discipline.

7-10 years of specialized experience in Data Engineering.

Preferred background in the Retail sector, with an understanding of industry‑specific data challenges.

Technical Leadership

Lead the DE Team/Owning the DE tasks (Design, build, maintain entire pipelines)

Data quality/Integrity/availability across the system