Senior Azure Data Engineer

ESP Engineered; Hyderabad; On-site; INR 1,000,000 - 1,500,000; Full time

Job summary

ESP Engineered in Hyderabad, India is seeking an Azure Data Engineer to design, build, and maintain data solutions on Microsoft Azure. The role involves crafting ETL processes, managing databases, and collaborating with cross-functional teams to enhance decision-making capabilities.

The ideal candidate has 4+ years of experience in data engineering, expertise in SQL and MySQL, and familiarity with programming languages like Python or Spark. This position emphasizes data quality and governance practices.

Qualifications

• 4+ years of experience in data engineering or related roles.
• Extensive experience with SQL and MySQL, including database design and optimization.
• Understanding of data warehousing concepts and big data technologies is a plus.

Responsibilities

• Design and implement ETL processes using Azure Data Factory and Azure Databricks.
• Develop and maintain relational databases using SQL and MySQL.
• Collaborate with data scientists and business stakeholders to ensure data quality.

Skills

Data Pipeline Development

Database Management

SQL Optimization

Data Governance

Python

Collaboration

Tools

Azure Data Factory

Azure Databricks

MySQL

Job description

Job Summary

As an Azure Data Engineer, you will design, build, and maintain data solutions on Microsoft Azure, leveraging your expertise in SQL and MySQL to ensure high-quality data availability for analysis and reporting. You will collaborate closely with cross-functional teams to meet data needs and enhance decision-making capabilities.

Key Responsibilities

• Data Pipeline Development: Design and implement ETL processes using Azure Data Factory and Azure Databricks, ensuring data integration from various sources.
• Database Management: Develop and maintain relational databases using SQL and MySQL, ensuring optimal performance and reliability.
• Data Modelling: Create and maintain data models to support analytical needs, ensuring data is structured for efficient querying and reporting.
• Query Optimization: Write and optimize complex SQL queries for data retrieval and reporting, improving performance and efficiency.
• Collaboration: Work with data scientists, analysts, and business stakeholders to understand data requirements and ensure data quality.
• Security and Compliance: Implement data governance practices and ensure compliance with data protection regulations.
• Documentation: Create and maintain comprehensive documentation for data pipelines, databases, and architectural designs.

Qualifications

• 4+ years of experience in data engineering or related roles.
• Familiarity with AWS services will be advantageous in our multi-cloud environment.
• Extensive experience with SQL and MySQL, including database design, optimization, and data manipulation.
• Familiarity with programming languages such as Python or Spark for data processing and analysis.
• Understanding of data warehousing concepts and big data technologies (e.g., Hadoop, Spark) is a plus.
• Experience with data governance and data quality best practices.
• Excellent problem-solving skills and attention to detail.
• Strong communication skills and ability to work collaboratively in a team environment.

Preferred Qualifications

• Experience with real-time data processing and stream analytics.
• Knowledge of machine learning concepts and tools.