Senior Azure Data Engineer

USEReady Inc.; Bengaluru; Hybrid; INR 2,600,000 - 3,800,000; Full time

Job summary

USEReady Inc. in Bengaluru, India seeks a Senior Azure Data Engineer to design, build, and optimize scalable data pipelines on Azure Databricks and ADF, enabling analytics and BI workloads.

You will implement data governance, Delta Lake models, and data lakehouse architectures, collaborating with stakeholders to deliver performant, cost-efficient data solutions.

The role requires 6+ years of experience, strong Python/PySpark, and a track record of improving data reliability and throughput.

Qualifications

• 6+ years of experience in Data Engineering, with strong Azure-based data platforms.
• Hands-on experience with Azure Databricks.
• Expertise in Databricks Notebooks using Python and SQL.
• Strong experience with Databricks Unity Catalog, governance, security, and access control.
• Hands-on experience with Databricks performance optimization and cost/compute optimization.
• Strong experience with Azure Data Factory (ADF) and ETL/ELT orchestration.
• Advanced Python and PySpark programming skills.
• Strong understanding of Apache Spark architecture, internals, and performance tuning.
• Strong SQL skills, including complex queries and large-volume data processing.
• Strong understanding of data warehousing concepts and data modeling (Star/Snowflake).
• Experience with Delta Lake and modern lakehouse architecture.
• Good knowledge of Azure Data Lake Storage (ADLS).
• Experience with Azure Synapse Analytics and Azure SQL Database.
• Understanding of data governance, data security, and data lineage.
• Experience with large-scale distributed data processing environments.
• Strong problem-solving, communication, and collaboration skills.

Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines using Azure Databricks and Azure Data Factory (ADF).
• Develop data processing solutions using Python, PySpark, SQL, and Apache Spark.
• Build and maintain Delta Lake tables and scalable data models for analytics and BI workloads.
• Implement and manage Databricks Unity Catalog for data governance, security, access control, and data discovery.
• Optimize Databricks jobs, Spark workloads, SQL queries, and data pipelines for performance and cost efficiency.
• Apply Apache Spark internals and tuning techniques to improve distributed processing performance.
• Develop complex SQL queries with efficient joins for large datasets.
• Design and implement ETL/ELT workflows using Azure Data Factory.
• Work with Azure data services including ADLS, Azure Synapse Analytics, and Azure SQL Database.
• Implement data quality checks, validation, reconciliation, and monitoring.
• Implement data governance, security, access control, and data lineage across platforms.
• Troubleshoot pipeline failures and ensure high availability of data solutions.
• Collaborate with stakeholders to translate requirements into effective data solutions.
• Automate manual processes and improve data engineering workflows.
• Mentor junior data engineers and promote best practices in Azure and Databricks.

Skills

Azure Databricks

PySpark

Python

SQL

Databricks Unity Catalog

Azure Data Factory (ADF)

Spark tuning

Data governance

Delta Lake

Azure Synapse

Azure Data Lake Storage

Large-scale data processing

Education

Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related field

Tools

Databricks Notebooks

Job description

About USEREADY

USEReady helps enterprises apply AI and agentic intelligence to improve decisions, automate operations, and build smarter, more autonomous business systems.

For more than a decade, we have built the foundations that make this possible by modernizing BI environments, migrating legacy platforms, improving data quality, and enabling governed, cloud-first architectures. These foundations now support the next step: AI-driven insights, automated intelligence, and agent-powered decision support that reduce complexity and accelerate outcomes.

We work closely with technology leaders such as AWS, Elementum, Snowflake, Tableau, Databricks, and others to help organizations modernize analytics, strengthen governance, and deploy agentic automation with confidence. We founded in 2011 and Headquartered in New York City with 450+ experts across the United States, Canada, India, and Singapore, we serve industries including financial services, healthcare, manufacturing, government, education, and retail. Our deep expertise, player-coach delivery model, and focus on fast, measurable results make us a trusted partner for building an AI-ready enterprise.

About the Role

We are looking for an experienced Senior Azure Data Engineer with strong hands-on expertise in Databricks, Azure Data Factory (ADF), PySpark, Python, and SQL. The ideal candidate will be responsible for designing, developing, optimizing, and maintaining scalable data engineering solutions on Azure.

You will work with modern Azure data technologies to build high-performance data pipelines, implement data governance, optimize Databricks workloads, and deliver reliable data platforms that support analytics and business intelligence initiatives.

Key Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines using Azure Databricks and Azure Data Factory (ADF).

• Develop data processing solutions using Python, PySpark, SQL, and Apache Spark.

• Build and maintain Delta Lake tables and scalable data models for analytics and BI workloads.

• Implement and manage Databricks Unity Catalog for data governance, security, access control, and data discovery.

• Optimize Databricks jobs, Spark workloads, SQL queries, and data pipelines for performance, scalability, and cost efficiency.

• Apply Apache Spark internals and tuning techniques to improve distributed data processing performance.

• Develop complex and optimized SQL queries, including efficient joins and processing of large datasets.

• Design and implement ETL/ELT workflows using Azure Data Factory.

• Work with Azure data services including Azure Data Lake Storage (ADLS), Azure Synapse Analytics, and Azure SQL Database.

• Implement data quality checks, validation, reconciliation, monitoring, and error-handling mechanisms.

• Implement data governance, security, access control, and data lineage across data platforms.

• Troubleshoot pipeline failures and performance issues and ensure high availability and reliability of data solutions.

• Collaborate with business stakeholders, data architects, analysts, and engineering teams to understand requirements and deliver effective data solutions.

• Automate manual processes and continuously improve data engineering workflows and operational efficiency.

• Mentor junior data engineers and promote best practices across Azure and Databricks data engineering.

Required Skills & Expertise

• 6+ years of experience in Data Engineering, with strong experience in Azure-based data platforms.

• Strong hands-on experience with Azure Databricks.

• Expertise in Databricks Notebooks using Python and SQL.

• Strong experience with Databricks Unity Catalog, governance, security, and access control.

• Hands-on experience with Databricks performance optimization and cost/compute optimization.

• Strong experience with Azure Data Factory (ADF) and ETL/ELT orchestration.

• Advanced Python and PySpark programming skills.

• Strong understanding of Apache Spark architecture, internals, and performance tuning.

• Strong SQL skills, including complex queries, query optimization, joins, aggregations, and large-volume data processing.

• Strong understanding of data warehousing concepts and data modeling, including Star and Snowflake schemas.

• Experience with Delta Lake and modern lakehouse architecture.

• Good knowledge of Azure Data Lake Storage (ADLS).

• Experience with Azure Synapse Analytics and Azure SQL Database.

• Understanding of data governance, data security, access management, and data lineage.

• Experience working with large-scale distributed data processing environments.

• Strong problem-solving, communication, and collaboration skills.

Preferred Qualifications

• Databricks certifications are a plus.

• Microsoft Azure Data Engineering certifications are a plus.

• Experience working with enterprise-scale Azure data platforms.

• Experience implementing data quality, governance, and security frameworks.

• Experience mentoring or technically guiding junior data engineers.

Education

• Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related field is preferred.