Senior Technical Lead

USEReady; Bengaluru; On-site; INR 3,500,000 - 6,000,000; Full time

Job summary

USEReady in Bengaluru seeks a Senior Technical Lead - Azure Data Engineer with 8-10 years of experience to design scalable data pipelines and Databricks environments. You will implement Unity Catalog governance, Delta Lake models, and secure data workflows using ADLS and Azure Synapse integration.

You will mentor teams, optimize Spark workloads, and collaborate with stakeholders to deliver enterprise-grade analytics solutions across cloud data platforms.

Qualifications

• 8-10 years of data engineering experience with strong focus on Azure.
• Strong hands-on experience with Azure Databricks.
• Excellent experience with Python and PySpark.
• Strong knowledge of Apache Spark architecture, internals, and performance tuning.
• Strong SQL skills, including complex queries and large-scale data processing.
• Strong experience with Azure Data Factory (ADF).
• Hands-on experience with Databricks Notebooks using Python and SQL.
• Experience with Unity Catalog, data governance, security, and access control.
• Strong understanding of Delta Lake.
• Experience with Azure Data Lake Storage (ADLS).
• Good understanding of data warehousing and data modeling, including Star and Snowflake schemas.

Responsibilities

• Architect and design scalable Azure Databricks environments and data engineering solutions.
• Design and implement robust, scalable data pipelines using Databricks and Azure Data Factory (ADF).
• Develop Databricks notebooks using Python, PySpark, and SQL.
• Implement and manage Unity Catalog for data governance, security, access control, and data discovery.
• Design data models and define schema strategies, data structures, and pipeline organization.
• Build and maintain Delta Lake tables and data models supporting analytics and BI workloads.
• Optimize Databricks jobs, Spark workloads, SQL queries, and data pipelines for performance, scalability, and cost efficiency.
• Troubleshoot pipeline failures and performance issues and ensure reliability and uptime.
• Collaborate with stakeholders to understand business requirements and translate them into scalable data solutions.
• Identify opportunities to automate manual processes and improve overall data platform efficiency.
• Mentor junior and mid-level engineers and establish best practices across Azure and Databricks development.

Skills

Azure

Azure Databricks

Python

PySpark

SQL

Delta Lake

Unity Catalog

Azure Data Factory

ADLS

Spark tuning

Data modeling

Data warehousing

Mentoring

Stakeholder management

Tools

Databricks Notebooks

Python

SQL

Job description

USEReady helps enterprises apply AI and agentic intelligence to improve decisions, automate operations, and build smarter, more autonomous business systems.

For more than a decade, we have built the foundations that make this possible by modernizing BI environments, migrating legacy platforms, improving data quality, and enabling governed, cloud-first architectures. These foundations now support the next step: AI-driven insights, automated intelligence, and agent-powered decision support that reduce complexity and accelerate outcomes.

We work closely with technology leaders such as AWS, Elementum, Snowflake, Tableau, Databricks, and others to help organizations modernize analytics, strengthen governance, and deploy agentic automation with confidence. We founded in 2011 and Headquartered in New York City with 450+ experts across the United States, Canada, India, and Singapore, we serve industries including financial services, healthcare, manufacturing, government, education, and retail. Our deep expertise, player-coach delivery model, and focus on fast, measurable results make us a trusted partner for building an AI-ready enterprise.

Job Summary:

We are looking for a Senior Technical Lead - Azure Data Engineer with 8-10 years of experience and strong hands-on expertise in Microsoft Azure, Databricks, ADF, Python, PySpark, and SQL.

The ideal candidate will have strong experience designing and implementing scalable data engineering solutions, architecting Databricks environments, optimizing data pipelines and workloads, and implementing data governance and security using Unity Catalog.

The candidate should be comfortable working with large-scale distributed data platforms, mentoring engineering teams, and collaborating with business and technical stakeholders to deliver high-quality data solutions.

Key Responsibilities:

• Architect and design scalable Azure Databricks environments and data engineering solutions.
• Design and implement robust, scalable, and optimized data pipelines using Databricks and Azure Data Factory (ADF).
• Develop Databricks notebooks using Python, PySpark, and SQL.
• Implement and manage Unity Catalog for data governance, security, access control, and data discovery.
• Design data models and define schema strategies, data structures, and pipeline organization.
• Build and maintain Delta Lake tables and data models supporting analytics and BI workloads.
• Optimize Databricks jobs, Spark workloads, SQL queries, and data pipelines for performance, scalability, and cost efficiency.
• Apply knowledge of Apache Spark internals to troubleshoot and tune distributed data processing workloads.
• Develop complex and optimized SQL queries, including efficient joins and processing of large datasets.
• Implement data quality, monitoring, validation, and reconciliation mechanisms.
• Work with Azure data services including Azure Data Lake Storage (ADLS), Azure Synapse Analytics, and Azure SQL Database.
• Implement data governance, security, access controls, and data lineage.
• Troubleshoot pipeline failures and performance issues and ensure reliability and uptime.
• Collaborate with stakeholders to understand business requirements and translate them into scalable data solutions.
• Identify opportunities to automate manual processes and improve overall data platform efficiency.
• Mentor junior and mid-level engineers and establish best practices across Azure and Databricks development.

Mandatory Skills:

• 8-10 years of overall experience in Data Engineering, with strong focus on Azure.
• Strong hands-on experience with Azure Databricks.
• Excellent experience with Python and PySpark.
• Strong knowledge of Apache Spark architecture, internals, and performance tuning.
• Strong SQL skills, including complex queries, query optimization, and large-scale data processing.
• Strong experience with Azure Data Factory (ADF).
• Hands-on experience with Databricks Notebooks using Python and SQL.
• Experience with Unity Catalog, data governance, security, and access control.
• Strong understanding of Delta Lake.
• Experience with Azure Data Lake Storage (ADLS).
• Good understanding of data warehousing and data modeling, including Star and Snowflake schemas.
• Experience working with large-scale distributed data processing environments.
• Strong troubleshooting and performance optimization skills.

Good to Have:

• Experience with Azure Synapse Analytics.
• Experience with Azure SQL Database.
• Databricks or Azure Data Engineering certifications.
• Experience implementing enterprise-level data governance and data lineage.
• Experience mentoring technical teams and driving engineering best practices.
• Strong stakeholder management and communication skills.
• Alteryx Workflow experience
• Understanding the Finance Data Models & Databricks Apps