Data Engineering Assistant Manager

Kavi Software Technologies; Chennai; On-site; INR 1,800,000 - 3,400,000; Full time

Job summary

Kavi Software Technologies in Chennai seeks a Data Engineer- Assistant Manager to design and optimize scalable data pipelines powering analytics, reporting, and ML initiatives. You will own the full lifecycle from ingestion to delivery and mentor junior engineers.

You will build ETL/ELT processes, optimize SQL, and craft Power BI data models and dashboards connected to enterprise data pipelines. Strong DW and RBAC experience required.

Qualifications

• 10+ years in data engineering with ETL/ELT pipelines.
• SQL and Python proficiency.
• Experience with Airflow, dbt, Informatica, Talend or SSIS.
• Cloud platforms: Redshift, Glue, ADF, Synapse, BigQuery.
• Relational and big data: PostgreSQL/MySQL/SQL Server; Spark/Hadoop/Hive.
• DW concepts, dimensional modeling, data architecture.
• Power BI knowledge: data models, DAX, dashboards.
• Power BI performance: incremental refresh, aggregations, Import vs Direct Query.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines ingesting data from diverse sources.
• Build and optimize data models (star/schema) for warehouses and lakes for BI use.
• Own end-to-end orchestration, monitoring, and error handling for pipelines.
• Optimize SQL queries and pipeline performance on large datasets.
• Collaborate with BI teams to design semantic layers and Power BI datasets.
• Develop Power BI data models, DAX measures, and dataset refresh pipelines.
• Optimize Power BI performance: incremental refresh, aggregations, Import vs DirectQuery.
• Implement data quality checks and observability for pipelines.
• Manage CI/CD for data pipelines and Power BI deployments.
• Ensure data governance and row-level security across pipelines/reports.
• Mentor junior engineers and contribute to engineering best practices.
• Troubleshoot production pipeline and reporting issues to minimize downtime.

Skills

ETL pipelines

SQL

Python

Airflow

dbt

Informatica

Talend

SSIS

Spark

Hadoop

Power BI

DAX

Data warehouse

Dimensional modeling

Tools

AWS Redshift

Azure Data Factory

GCP BigQuery

Job description

About the Role

We're looking for a Data Engineer- Assistant Manager to design, build, and optimize scalable data pipelines that power analytics, reporting, and machine learning initiatives across the organization. You'll own the full lifecycle of data pipeline development from ingestion to transformation to delivery while also enabling downstream BI consumption through well-structured, reporting-ready datasets. You'll mentor junior engineers and drive best practices in data engineering.

Key Responsibilities

• Design, develop, and maintain robust, scalable ETL/ELT pipelines to ingest data from diverse sources (databases, APIs, flat files, streaming sources)
• Build and optimize data models (star/snowflake schemas) for data warehouses and data lakes, structured for efficient BI consumption
• Own end-to-end pipeline orchestration, monitoring, and error handling to ensure high reliability and data quality
• Optimize SQL queries and pipeline performance for large-scale datasets
• Partner with BI developers and business stakeholders to design semantic layers and datasets that support Power BI dashboards and reports
• Build and maintain Power BI data models (star schema), DAX measures, and dataset refresh pipelines, ensuring alignment with underlying ETL structures
• Optimize Power BI datasets and queries for performance, including incremental refresh strategies and efficient data source connections (Import vs. DirectQuery)
• Implement data quality checks, validation frameworks, and observability/monitoring for pipelines
• Manage and evolve CI/CD practices for data pipeline (and where applicable, Power BI deployment pipeline) releases
• Ensure data governance, security, and row-level security (RLS) standards are met across pipelines and Power BI reports
• Mentor junior data engineers and contribute to engineering best practices and documentation
• Troubleshoot and resolve production pipeline and reporting issues, ensuring minimal downtime

Re-quired Skills & Qualifications

• 10+ years of hands-on experience in data engineering with a strong focus on ETL/ELT pipeline development
• Strong proficiency in SQL and at least one programming language (Python preferred)
• Hands-on experience with ETL/orchestration tools such as Apache Airflow, dbt, Informatica, Talend, or SSIS
• Solid experience with cloud data platforms (AWS Redshift/Glue, Azure Data Factory/Synapse, or GCP BigQuery/Dataflow)
• Experience working with both relational databases (PostgreSQL, MySQL, SQL Server) and big data technologies (Spark, Hadoop, Hive)
• Strong understanding of data warehousing concepts, dimensional modeling, and data architecture principles
• Working knowledge of Power BI building data models, writing DAX, and designing dashboards/reports connected to enterprise data pipelines
• Understanding of Power BI performance optimization (incremental refresh, aggregations, query folding, Import vs. Direct Query trade-offs)
• Experience with data pipeline orchestration, scheduling, and monitoring frameworks
• Familiarity with version control (Git) and CI/CD pipelines for data engineering workflows