Analytics Consultant

Solvex Solutions; Bengaluru; Hybrid; INR 4,000,000 - 7,000,000; Full time

Job summary

Solvex Solutions in Bengaluru is seeking experienced Azure Databricks Data Engineers and technical leads/architects to design, build, and maintain scalable data pipelines on the Azure cloud.

You'll work with PySpark, Delta Lake, Python, and Advanced SQL to implement lakehouse architectures, ensure data quality, and optimize performance while guiding junior engineers. Collaborate with stakeholders and use Azure services like ADLS Gen2, ADF, and Key Vault to deliver robust data platforms.

Qualifications

• 6-12 years of IT experience with 5+ years hands-on Databricks
• Strong expertise in Azure Databricks, PySpark, Delta Lake, Python, and Advanced SQL
• Hands-on experience developing ETL/ELT pipelines and dimensional data modelling
• Good understanding of Azure data services: ADLS Gen2, ADF, Key Vault
• Strong knowledge of Spark and SQL performance tuning
• Experience with data quality, validation, and CDC
• Lead/Architect candidates: Lakehouse architecture, Unity Catalog, end-to-end technical ownership, governance

Responsibilities

• Design, develop, and maintain reliable, scalable ETL/ELT data pipelines
• Build and optimize data processing solutions on Azure Databricks
• Implement dimensional data modelling and efficient transformation strategies
• Optimize Spark jobs, SQL queries, and pipelines for performance and cost
• Ensure data quality, validation, consistency, and CDC implementation
• Collaborate with architects, product owners, and stakeholders on data solutions
• Own end-to-end solution design, Lakehouse architecture, and production support
• Establish data governance, security, and access control with Unity Catalog
• Drive performance tuning and cost optimization across Databricks, Spark, SQL, PostgreSQL
• Mentor team members, conduct code reviews, and promote engineering best practices
• Lead technical decisions to ensure scalable, secure data platforms

Skills

Databricks

PySpark

Python

Advanced SQL

Data modelling

Data governance

Mentoring

Tools

Azure Databricks

Delta Lake

Spark

PostgreSQL

Unity Catalog

ADF

ADLS Gen2

Key Vault

Job description

We are looking for experienced Azure Databricks Data Engineers and Technical Leads/Architects to design, develop, and maintain scalable data engineering solutions on Azure. The ideal candidates should have strong hands-on expertise in Databricks, PySpark, Delta Lake, Python, and Advanced SQL, along with experience in building enterprise-grade data pipelines and implementing Lakehouse architecture.

Key Responsibilities

• Design, develop, and maintain reliable, scalable ETL/ELT data pipelines for structured and semi-structured data.
• Build and optimize data processing solutions using Azure Databricks, PySpark, Delta Lake, Python, and Advanced SQL.
• Implement dimensional data modelling and efficient data transformation strategies.
• Optimize Spark jobs, SQL queries, and data pipelines for performance and cost efficiency.
• Ensure data quality, validation, consistency, and Change Data Capture (CDC) implementation.
• Work with Azure services, including ADLS Gen2, Azure Data Factory (ADF), and Azure Key Vault.
• Collaborate with architects, product owners, and business stakeholders to deliver effective data solutions.

Additional Responsibilities

• Own end-to-end solution design, Lakehouse architecture, technical standards, and production support.
• Establish data governance, security, and access control practices using Unity Catalog.
• Drive performance tuning and cost optimization across Databricks, Spark, SQL, and PostgreSQL workloads.
• Conduct code reviews, mentor team members, and promote engineering best practices.
• Lead technical decisions and ensure scalable, secure, and maintainable enterprise data platforms.

Mandatory Skills

• 6-12 years of overall IT experience, including 5+ years of hands-on Databricks experience.
• Strong expertise in Azure Databricks, PySpark, Delta Lake, Python, and Advanced SQL.
• Hands-on experience developing ETL/ELT pipelines and implementing dimensional data modelling.
• Good understanding of Azure data services: ADLS Gen2, ADF, and Key Vault.
• Strong knowledge of Spark and SQL performance tuning.
• Experience with data quality, validation, and CDC.
• For Lead/Architect candidates: Lakehouse architecture, Unity Catalog, end-to-end technical ownership, and data governance.

Good to Have

• Experience with Delta Live Tables, Databricks Workflows, and Unity Catalog.
• Knowledge of CI/CD practices, Git, and automated deployment.
• Exposure to dashboarding tools such as Power BI or Apache Superset.
• Experience with PostgreSQL performance optimization.