Azure Data Engineer (Senior)

Infogain; Bengaluru; In loco; INR 1.000.000 - 1.500.000; Tempo pieno

Descrizione del lavoro

A technology and software engineering company is seeking an Azure Data Engineer. The ideal candidate will possess strong SQL and Python skills, with 6-9 years of experience in data quality or data engineering. Responsibilities include designing automated data quality rules and collaborating with teams to enhance data reliability. This full-time position is based in Bengaluru, Karnataka, and offers opportunities to work with advanced data platforms and tools in a collaborative environment.

Competenze

• 6–9 years of experience in data quality or data engineering.
• Proven experience with Ataccama DQ rule creation and monitoring.
• Hands-on exposure to Databricks for building and running data pipelines.

Mansioni

• Design and implement automated data quality rules.
• Build and operationalize data quality workflows.
• Conduct root cause analysis for data anomalies.

Conoscenze

SQL

Python

Data profiling

Problem-solving

Analytical thinking

Formazione

Bachelor’s degree in Computer Science or related field

Strumenti

Databricks

PySpark

Ataccama

Apache Airflow

Azure

Descrizione del lavoro

Join to apply for the Azure Data Engineer (Senior) role at Infogain.

Responsibilities

• Design and implement automated data quality rules and validation checks using Databricks (Delta Lake) and PySpark.
• Build and operationalize data quality workflows in Ataccama ONE / Ataccama Studio.
• Perform data profiling, anomaly detection, and reconciliation across systems and data sources.
• Establish thresholds, KPIs, and alerts for data quality metrics.
• Conduct root cause analysis and issue management for data anomalies.
• Investigate data quality incidents using SQL, Python, and Ataccama diagnostics.
• Collaborate with data engineers and business analysts to identify and remediate root causes.
• Document recurring data issues and contribute to preventive automation solutions.
• Partner with data stewards, governance, and analytics teams to define and maintain DQ rules and SLAs.
• Contribute to metadata enrichment, lineage documentation, and data catalog integration.
• Support the adoption of DQ frameworks and promote data reliability best practices.
• Integrate DQ validations into orchestration tools (Airflow, Databricks Workflows, or Azure Data Factory).
• Leverage Python/Pyspark libraries to complement existing platforms.
• Propose process improvements to enhance automation, monitoring, and exception management.

Tools & Platforms

• Databricks (Delta Lake)
• PySpark
• SQL
• Python
• Ataccama ONE / Ataccama Studio (DQ rule creation, profiling, workflows)
• Apache Airflow, Azure Data Factory, or Databricks Jobs (orchestration)
• Git, GitHub Actions, Azure DevOps (Version Control / CI‑CD)
• Collibra, Alation, Ataccama Catalog (Data Catalog / Lineage – optional)
• Azure (preferred), AWS, or GCP (Cloud environments)

Qualifications & Experience

• Bachelor’s degree in Computer Science, Information Systems, Statistics, or a related field.
• 6–9 years of experience in data quality, data engineering, or analytics operations.
• Strong command of SQL, Python, and PySpark for data validation and troubleshooting.
• Proven experience with Ataccama DQ rule creation and monitoring.
• Hands‑on exposure to Databricks for building and running data pipelines.
• Working knowledge of reconciliation processes, data profiling, and DQ metrics.

Soft Skills & Attributes

• Analytical thinker with strong problem‑solving abilities.
• Detail‑oriented and methodical approach to troubleshooting.
• Strong communication skills for cross‑functional collaboration.
• Proactive mindset, capable of owning issues through resolution.
• Comfortable balancing hands‑on technical work with business stakeholder interaction.

Preferred / Nice to Have

• Exposure to data governance frameworks or MDM initiatives.
• Familiarity with observability tools such as Grafana, Datadog, Prometheus.
• Understanding of CI/CD practices for data quality deployment.
• Certification in Databricks, Ataccama, or a major cloud platform (Azure/AWS).

Success Indicators

• Increase in automated data quality coverage across critical datasets.
• Reduction in recurring manual DQ exceptions.
• Improved timeliness and accuracy of data available for analytics.
• Positive stakeholder feedback on data trust and reliability.

Experience Level

• 6‑8 Years

About the Company

Infogain is a human‑centered digital platform and software engineering company based out of Silicon Valley. We engineer business outcomes for Fortune 500 companies and digital natives in the technology, healthcare, insurance, travel, telecom, and retail & CPG industries using technologies such as cloud, microservices, automation, IoT, and artificial intelligence. Infogain accelerates experience‑led transformation in the delivery of digital platforms. Infogain is also a Microsoft (NASDAQ: MSFT) Gold Partner and Azure Expert Managed Services Provider (MSP). We have offices in California, Washington, Texas, the UK, the UAE, and Singapore, with delivery centers in Seattle, Houston, Austin, Kraków, Noida, Gurgaon, Mumbai, Pune, and Bengaluru.

Seniority Level

• Not Applicable

Employment Type

• Full‑time

Job Function

• Information Technology

Industries

• IT Services and IT Consulting, Insurance, and Retail