Data Engineer

Arrow Electronics India Pvt Ltd; Maan; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Arrow Electronics India Pvt Ltd seeks a Data Engineer to design, develop and maintain ETL pipelines using Azure Data Factory, Databricks and PySpark for enterprise data warehouses. The role requires strong SQL skills, Azure DB experience, and solid data governance practices.

You will collaborate with cross-functional teams, ensure data quality, and contribute to Power BI reporting for executive-level insights in a full-time on-site setting in Ahmedabad. Prior Agile experience is preferred.

Qualifications

• Should be well-versed in design and development of ETL solutions using Azure Data Factory, Azure Databricks, PySpark and SQL for large-scale data warehouses.
• Must work collaboratively with team members and users for operational support.
• Expertise in Azure Data Factory, Databricks, PySpark, Azure SQL Database, SQL Stored Procedures and performance tuning.
• Excellent communication skills with strong business and data analysis abilities.
• Experience in Agile methodologies.
• Experience in developing and working on Power BI reports for enterprise level reports.

Responsibilities

• Develop, fine-tune, automate, and maintain ETL pipelines using Azure Data Factory, Databricks, PySpark, Azure SQL DB, and SQL Server Stored Procedures.
• Develop ETL solutions using data integration tools interfacing between source applications and the Enterprise Data Warehouse.
• Document ETL data mappings, data dictionaries, processes, programs, and solutions per data governance standards.
• Integrate ETL development with existing projects to maximize object reuse.
• Create, execute, and document unit test plans for ETL and data integration processes.
• Collaborate with data team peers to assess and troubleshoot data quality at intake points.
• Troubleshoot data issues and defects to determine root causes and perform performance tuning.

Skills

ETL design

Agile methodologies

Data analysis

Communication

Collaboration

Tools

Azure Data Factory

Azure Databricks

Azure SQL Database

SQL

Python (PySpark)

Power BI

SQL Server Stored Procedures

Job description

Data Engineer Position: Data Engineer Job Description: JD For Data Engineer Experience: 5 Years Tools: Azure Data Factory, Azure Databricks, Azure SQL Database, SQL, Python (PySpark), Microsoft Power BI

Mandatory Skills

• Should be well-versed in the design and development of ETL solutions using Azure Data Factory, Azure Databricks, Python (PySpark), and SQL for large-scale data warehouses.
• Should be able to work collaboratively with other team members, as well as users for operational support
• Expertise in Azure Data Factory, Azure Databricks, Python (PySpark), Azure SQL Database, SQL Stored Procedure development, and performance tuning.
• Should possess very good communication with strong business and data analysis skills
• Experience in using Agile methodologies
• Experience in developing and working on Power BI reports for enterprise level reports.

Key Responsibilities

• Develop, fine-tune, automate, and maintain ETL pipelines using Azure Data Factory, Azure Databricks, Python (PySpark), Azure SQL Database, and SQL Server Stored Procedures.
• Develop ETL solutions using data integration tools for interfacing between source application and the Enterprise Data Warehouse
• Document ETL data mappings, data dictionaries, processes, programs, and solutions as per established standards for data governance
• Integrate ETL development with existing projects to maximize object reuse
• Create, execute, and document unit test plans for ETL and data integration processes and programs
• Work with the peers in data team to assess and troubleshoot potential data quality issues at key intake points such as validating control totals at intake.
• Troubleshoot data issues and defects to determine root cause
• Performance tuning of the ETL process and SQL queries, and recommend and implement ETL and query tuning techniques
• Develop and create transformation queries, views, and stored procedures for ETL processes, and process automations

Location: IN-GJ-Ahmedabad, India-Ognaj (eInfochips) Time Type: Full time Job Category: Engineering Services Experience Level Mid Level