Lead Data Engineer

Tranway Technologies Limited; Chennai; Hybrid; INR 3,000,000 - 6,000,000; Full time

Job summary

Tranway Technologies Limited in Chennai, India, seeks a technical leader to modernize the Sales Validation platform, defining the future-state architecture and guiding delivery across major workstreams.

You will own Databricks and Synapse migrations, set engineering standards, mentor engineers, review designs, and push for scalable, high-performance data pipelines using GenAI-assisted development.

Qualifications

• Lead Databricks architecture and delivery with a focus on modernizing the Sales Validation platform.
• Strong migration experience from Synapse to Databricks and other Azure data services.
• Establish engineering standards and reusable frameworks for scalable data platforms.
• Mentor engineers and conduct design reviews to ensure quality and maintainability.
• Own technical delivery across major workstreams with emphasis on performance and scalability.

Responsibilities

• Lead Databricks architecture and delivery
• Drive Synapse-to-Databricks migration
• Establish engineering standards and reusable frameworks
• Mentor engineers and conduct design reviews
• Own technical delivery across major workstreams
• Drive performance optimization and scalability

Skills

Databricks

PySpark

Spark SQL

Delta Lake

Unity Catalog

Databricks Workflows

Databricks Asset Bundles

Lakehouse Architecture

Azure Platform

Azure Data Factory

ADLS Gen2

Azure SQL

Azure Security

Migration Experience

Synapse Pipelines

Synapse Spark

Synapse SQL Pools

Synapse Notebooks

Synapse Data Flows

ETL Modernization

Engineering

Advanced SQL

Data Modeling

GitLab CI/CD

Performance Tuning

Code Reviews

GenAI

AI Coding Assistants

Prompt Engineering

LLM Fundamentals

Job description

Role Summary

Technical leader responsible for the modernization and future-state architecture of the Sales Validation platform.

Responsibilities

• Lead Databricks architecture and delivery
• Drive Synapse-to-Databricks migration
• Establish engineering standards and reusable frameworks
• Mentor engineers and conduct design reviews
• Own technical delivery across major workstreams
• Drive performance optimization and scalability

Required Skills

• Databricks (Highest Priority)
• Databricks
• PySpark
• Spark SQL
• Delta Lake
• Unity Catalog
• Databricks Workflows
• Databricks Asset Bundles
• Lakehouse Architecture
• Azure Platform
• Azure Data Factory (ADF)
• ADLS Gen2
• Azure SQL
• Azure Security Patterns
• Migration Experience
• Synapse Pipelines
• Synapse Spark
• Synapse SQL Pools
• Synapse Notebooks
• Synapse Data Flows
• Modernization of legacy ETL workloads
• Engineering
• Advanced SQL
• Data Modeling
• GitLab CI/CD
• Performance Tuning
• Code Reviews
• GenAI
• Hands-on experience with GenAI coding assistants (Claude Code, GitHub Copilot, or similar) for AI-assisted development.
• Prompt engineering skills writing clear instructions and feedback to guide AI toward desired output.
• Ability to critically review and validate AI-generated code and content before use.
• Basic understanding of LLM concepts — RAG, context windows, hallucinations, and responsible AI practices.

Preferred

• Apache Airflow
• Snowflake
• Data Governance
• Enterprise-scale Azure platform implementation