Senior Data Engineer

Arrow Electronics India Pvt Ltd; Maan; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

eInfochips is seeking a Data Engineer - Senior Engineer(Level 1) to design and implement ETL pipelines on Azure data services. You will work closely with cross-functional teams to deliver enterprise-scale data solutions.

The role requires 5 years of experience with ADF, Databricks, PySpark, SQL, and Power BI, and a strong focus on data quality, governance, and performance tuning. Pune-based, full-time opportunity.

Qualifications

• ETL design/development using ADF, Databricks, PySpark and SQL for large warehouses.
• Collaborate with team members and users for operational support.
• Expertise in ADF, Databricks, PySpark, Azure SQL DB, SQL stored procedures, and tuning.
• Strong communication with business and data analysis skills.
• Experience with Agile methodologies.
• Experience developing Power BI enterprise reports.

Responsibilities

• Develop, tune, automate, and maintain ETL pipelines using ADF/Databricks/PySpark/SQL.
• Create ETL solutions interfacing between source apps and the Data Warehouse.
• Document ETL mappings, data dictionaries, and governance processes.
• Integrate ETL with existing projects to maximize reuse.
• Create and execute unit tests for ETL and data integration.
• Collaborate with data team to address data quality at intake points.
• Troubleshoot data issues and determine root causes.
• Tune ETL and SQL performance and apply optimization techniques.
• Develop transformation queries, views, and stored procedures for ETL.

Skills

Azure Data Factory

Azure Databricks

Python (PySpark)

Azure SQL Database

SQL

Power BI

Agile methodologies

Tools

Azure Data Factory

Azure Databricks

SQL Server

Job description

Data Engineer - Senior Engineer(Level 1) Position: Data Engineer - Senior Engineer(Level 1) Job Description: JD For Data Engineer Experience: 5 Years Tools: Azure Data Factory, Azure Databricks, Azure SQL Database, SQL, Python (PySpark), Microsoft Power BI

Mandatory Skills:

• Should be well-versed in the design and development of ETL solutions using Azure Data Factory, Azure Databricks, Python (PySpark), and SQL for large-scale data warehouses.
• Should be able to work collaboratively with other team members, as well as users for operational support
• Expertise in Azure Data Factory, Azure Databricks, Python (PySpark), Azure SQL Database, SQL Stored Procedure development, and performance tuning.
• Should possess very good communication with strong business and data analysis skills
• Experience in using Agile methodologies
• Experience in developing and working on Power BI reports for enterprise level reports.

Key Responsibilities:

• Develop, fine-tune, automate, and maintain ETL pipelines using Azure Data Factory, Azure Databricks, Python (PySpark), Azure SQL Database, and SQL Server Stored Procedures.
• Develop ETL solutions using data integration tools for interfacing between source application and the Enterprise Data Warehouse
• Document ETL data mappings, data dictionaries, processes, programs, and solutions as per established standards for data governance
• Integrate ETL development with existing projects to maximize object reuse
• Create, execute, and document unit test plans for ETL and data integration processes and programs
• Work with the peers in data team to assess and troubleshoot potential data quality issues at key intake points such as validating control totals at intake.
• Troubleshoot data issues and defects to determine root cause
• Performance tuning of the ETL process and SQL queries, and recommend and implement ETL and query tuning techniques
• Develop and create transformation queries, views, and stored procedures for ETL processes, and process automations

Location: IN-MH-Pune, India-Blue Ridge-Hinjewadi (eInfochips)

Time Type: Full time

Job Category: Engineering Services

Experience Level: Senior Level