Reflections Info Systems - Data Engineering & Analytics Lead - SQL/PySpark

Vista Info Systems; Ernakulam; Presencial; INR 3 500 000 - 7 000 000; Tempo integral

Resumo da oferta

Vista Info Systems in India seeks a seasoned Data Engineering & Analytics professional with 10+ years of experience building data pipelines, data models and analytics solutions using Databricks, SQL, PySpark, Python and Power Design. Develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, Python, and Azure Data Factory.

Build and optimize enterprise-grade data solutions using Databricks Lakehouse architecture and Delta Lake.

Qualificações

• Over 10 years of data engineering/analytics experience.
• Hands-on with Databricks, SQL, PySpark, Python and Power BI.
• Experience building data pipelines, data models and analytics solutions.

Responsabilidades

• Build and optimize enterprise-grade data solutions on Delta Lake.
• Develop high-performance ETL/ELT pipelines with CI/CD.
• Create and maintain data models to support BI and reporting.
• Write and optimize complex SQL queries and data transformations.
• Design Medallion Architecture for analytics platforms.
• Deliver Power BI dashboards and analytics solutions.
• Collaborate with stakeholders, data architects, and product owners.
• Ensure data governance, metadata management, and data cataloging.

Conhecimentos

Databricks

SQL

PySpark

Python

ETL/ELT

Data Modeling

Power BI

Azure Data Factory

Medallion Architecture

AI analytics

Ferramentas

Azure Data Factory (ADF)

Git

CI/CD

Descrição da oferta de emprego

Introduction

We are looking for a strong Data Engineering & Analytics professional with 10+ years of experience in building data pipelines, data models and analytics solutions, with hands-on expertise in Databricks, SQL, PySpark, Python and Power Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, Python, and Azure Data Factory.

• Build and optimize enterprise-grade data solutions using Databricks Lakehouse architecture and Delta Lake.
• Develop high-performance data transformation workflows and implement data quality controls.
• Create and maintain logical and physical data models to support reporting, analytics, and business intelligence requirements.
• Write and optimize complex SQL queries, stored procedures, views, and data transformation logic.
• Design and implement Medallion Architecture (Bronze, Silver, Gold layers) for modern analytics platforms.
• Develop interactive dashboards, reports, and analytical solutions using Power BI.
• Collaborate with business stakeholders to understand reporting and analytics requirements.
• Ensure data governance, metadata management, and data cataloging best practices are followed.
• Implement performance optimization strategies across data pipelines, Databricks workloads, and reporting solutions.
• Leverage AI/LLM-enabled tools for data analysis, insight generation, and analytics acceleration.
• Work closely with cross-functional teams including Data Architects, Business Analysts, and Product Owners.
• Support CI/CD and version control practices for data engineering solutions.

Introduction

We are looking for a strong Data Engineering & Analytics professional with 10+ years of experience in building data pipelines, data models and analytics solutions, with hands-on expertise in Databricks, SQL, PySpark, Python and Power Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, Python, and Azure Data Factory.

• Build and optimize enterprise-grade data solutions using Databricks Lakehouse architecture and Delta Lake.
• Develop high-performance data transformation workflows and implement data quality controls.
• Create and maintain logical and physical data models to support reporting, analytics, and business intelligence requirements.
• Write and optimize complex SQL queries, stored procedures, views, and data transformation logic.
• Design and implement Medallion Architecture (Bronze, Silver, Gold layers) for modern analytics platforms.
• Develop interactive dashboards, reports, and analytical solutions using Power BI.
• Collaborate with business stakeholders to understand reporting and analytics requirements.
• Ensure data governance, metadata management, and data cataloging best practices are followed.
• Implement performance optimization strategies across data pipelines, Databricks workloads, and reporting solutions.
• Leverage AI/LLM-enabled tools for data analysis, insight generation, and analytics acceleration.
• Work closely with cross-functional teams including Data Architects, Business Analysts, and Product Owners.
• Support CI/CD and version control practices for data engineering solutions.

Primary Skills

• Databricks (Delta Lake, Lakehouse, data transformation, performance optimization)
• SQL (advanced queries, joins, window functions, performance tuning, data transformation)
• PySpark (transformations, joins, aggregations, window functions, deduplication, optimization)
• Python
• ETL / ELT Pipeline Development
• Data Modeling & Schema Design
• Power BI (data modelling, dashboards, analytics)
• Azure Data Factory (ADF)
• Data Lake / Lakehouse Architecture / Medallion Architecture
• AI‑assisted Data Analysis & Analytics

Secondary Skills

• ADLS Gen2
• Data Quality & Validation
• Git / Version Control
• CI/CD Fundamentals
• Data Governance / Metadata / Data Cataloguing
• Exposure to Process / Operational Analytics or Process Intelligence is a plus

(ref:hirist.tech)