Azure Data Engineer - SQL / Python

Forward Eye Technologies; Pune District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Forward Eye Technologies is seeking a Data Engineer to design, develop, and implement ETL pipelines using Azure technologies. You will collaborate with data engineers and scientists to optimize data solutions, ensure data cleansing and transformation, and automate deployment with CI/CD practices.

The role involves loading data into data warehouses, building integrated pipelines, and maintaining governance and security standards.

Qualifications

• 2-3 years of experience in Azure Data Analytics, Synapse, DevOps.
• Proficiency in Python, SQL, Py-Spark, Spark-SQL.
• Experience with Azure ADLS, Synapse, Stream Analytics, SQL DW, Analysis Services, Serverless Architecture, ARM Templates.
• Experience with cloud platforms like Azure, AWS.
• Knowledge of CI/CD pipelines, automation tools and Git.

Responsibilities

• Design, develop, and implement ETL pipelines for deployment and integration.
• Collaborate with data engineers and data scientists to optimize data solutions.
• Ensure data is cleansed, mapped, transformed for storage and use.
• Automate tasks and deploy production-grade code with tests and CI.
• Load transformed data into data warehouses and reporting structures.
• Build data pipelines to bring data together across systems.
• Extract data, troubleshoot and maintain the data warehouse.
• Implement best practices for pipeline governance and deployment performance.
• Ensure compliance with security and data privacy standards.
• Design solutions using Microsoft Azure services and other tools.

Skills

Azure Data Analytics

Synapse

DevOps

Python

SQL

PySpark

Spark SQL

Azure ADLS

Stream Analytics

SQL DW

Analysis Services

Serverless Architecture

ARM Templates

CI/CD

Git

AWS

Job description

Main Roles and Responsibilities:

• Design, develop, and implement ETL pipelines for the continuous deployment and integration.
• Collaborate with other data engineers and data scientists to understand data requirements and optimize data solution processes.
• Assure that data is cleansed, mapped, transformed, and otherwise optimised for storage and use according to business and technical requirements.
• The ability to automate tasks and deploy production standard code (with unit testing, continuous integration, versioning etc.).
• Load transformed data into storage and reporting structures in destinations including data warehouse, high speed indexes, real-time reporting systems and analytics applications.
• Build data pipelines to collectively bring data together.
• Other responsibilities include extracting data, troubleshooting and maintaining the data warehouse.
• Implement best practices for pipeline building and governance.
• Troubleshoot and resolve issues related to pipeline deployment and performance.
• Ensure compliance with security and data privacy standards.
• Solution design using Microsoft Azure services and other tools.

Required skills and experience:

• 2-3 years of experience in Azure Data Analytics, Synapse, DevOps.
• Proficiency in programming languages such as Python, SQL, Py-Spark, Spark-SQL.
• Experience with Azure: ADLS, Synapse, Stream Analytics, SQL DW, Analysis Services, Serverless Architecture, ARM Templates.
• Experience with cloud platforms like Azure, AWS or and their respective services.
• Knowledge of CI/CD pipelines, automation tools and version control systems like Git.
• Strong problem-solving skills and ability to troubleshoot complex issues.
• Experience with monitoring tools.
• Ability to work collaboratively in cross-functional teams.

Nice to have:

• Should have Data Engineer certification from any discipline - Azure, AWS.
• Azure Synapse Studio, Data bricks, Fabric.
• Familiarity with agile software development lifecycle (SCRUM, Kanban, etc.).

Location - Hyderabad,Ahmedabad,pune,chennai,kolkata.