Azure Data Engineer

Veraxion AG; Bengaluru; Presencial; INR 600.000 - 1.200.000; Jornada completa

Descripción de la vacante

Veraxion AG in Bengaluru is seeking a senior Data Engineer to architect and oversee data infrastructure, pipelines and data products that are scalable, modular and cost-efficient.

You will design, deploy and maintain automated ETL pipelines using Azure Data Factory, Synapse, Databricks and Fabric, ensuring data quality and governance. Mentoring juniors and collaborating with data scientists and analysts are key parts of the role.

Formación

• Bachelor’s degree in Computer Science or related field and strong programming fundamentals.
• At least 8 years of proven experience as a Data Engineer or similar role.
• Experience with SQL DML querying modern RDBMS (SQL Server, PostgreSQL).
• Experience with big data technologies (e.g., Spark).
• Strong problem-solving and collaboration skills.

Responsabilidades

• Oversee data infrastructure to ensure scalability, operation efficiency and resiliency.
• Design, develop, and maintain data pipelines and ETL processes.
• Utilize Azure data services for storage and processing.
• Mentor junior data engineers within the organization.
• Collaborate with data scientists, data analysts, data architects and stakeholders.
• Optimize data pipelines in Azure for performance and reliability.
• Ensure data quality and integrity through validation techniques.
• Develop and maintain documentation for data processes and configurations.
• Monitor and troubleshoot data pipeline issues and CI/CD for data solutions.

Conocimientos

Data pipelines

ETL pipelines

Azure

SQL

Big data

Spark

Data quality

Documentation

CI/CD

Mentoring

Collaboration

Programming fundamentals

Educación

Bachelor's degree in Computer Science or related field

Herramientas

Azure Data Factory

Azure Synapse

Azure Databricks

Azure Fabric

SQL Server

PostgreSQL

Spark

Descripción del empleo

We are looking for a Data Engineer who designs and oversees the data infrastructure, data products and data pipelines that are resilient to change, modular, flexible, scalable, reusable, and cost effective.

Utilizes software engineering principles to deploy and maintain fully automated data transformation pipelines that combine a large variety of storage and computation technologies to handle a distribution of data types and volumes in support of data architecture design.

• Oversee the data infrastructure to ensure scalability, operation efficiency and resiliency.
• Mentor junior data engineers within the organization.
• Design, develop, and maintain data pipelines and ETL processes using Microsoft Azure services (e.g., Azure Data Factory, Azure Synapse, Azure Databricks, Azure Fabric).
• Utilize Azure data storage accounts for organizing and maintaining data pipeline outputs. (e.g., Azure Data Lake Storage Gen 2 & Azure Blob storage).
• Collaborate with data scientists, data analysts, data architects and other stakeholders to understand data requirements and deliver high-quality data solutions.
• Optimize data pipelines in the Azure environment for performance, scalability, and reliability.
• Ensure data quality and integrity through data validation techniques and frameworks.
• Develop and maintain documentation for data processes, configurations, and best practices.
• Monitor and troubleshoot data pipeline issues to ensure timely resolution.
• Manage the CI/CD process for deploying and maintaining data solutions.
• Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent experience) and able to demonstrate high proficiency in programming fundamentals.
• At least 8 years of proven experience as a Data Engineer or similar role dealing with data and ETL processes.
• Experience utilizing SQL DML to query modern RDBMS in an efficient manner (e.g., SQL Server, PostgreSQL).
• Experience with big data technologies (e.g., Spark).
• Strong problem-solving skills and attention to detail.
• Excellent communication and collaboration skills.