Senior Data Engineer

Arrow Electronics India Pvt Ltd; Ahmedabad District; Hybrid; INR 2,800,000 - 4,200,000; Full time

Job summary

Arrow Electronics India Pvt Ltd in Ahmedabad, India, seeks a Senior Data Engineer to build scalable data pipelines on Azure, using Databricks and Delta Lake for analytics and ML. You will design and optimize ETL/ELT workflows, enforce data quality, and publish curated datasets across analytics platforms.

The role requires strong Python and SQL, experience with CI/CD and governance tooling, and collaboration in an Agile environment. Hybrid work and cloud-first data security are emphasized.

Qualifications

• Hands-on experience building production pipelines with Azure Data Factory and Databricks.
• Knowledge of Medallion Architecture and Delta Lake.
• Strong Python (pandas/PySpark) and SQL.
• Git workflow; CI/CD integration (Azure DevOps/GitHub Actions/Jenkins).
• Packaging reusable code and configuration-driven jobs.
• Data modeling and warehousing with dimensional modeling and Synapse/Snowflake/SQL Server.
• Data quality checks and monitoring; drift detection and pipeline observability.
• Azure ADLS Gen2, Key Vault, Databricks, ADF basics.
• Purview/Unity Catalog RBAC and secure data sharing; PII handling.

Responsibilities

• Design, build, and maintain ETL/ELT pipelines in Azure Data Factory and Databricks across Bronze - Silver - Gold layers.
• Implement Delta Lake best practices (ACID, schema evolution, MERGE/upsert, time travel, Z-ORDER).
• Write performant PySpark and SQL; tune jobs (partitioning, caching, join strategies).
• Create reusable components; manage code in Git; contribute to CI/CD pipelines (Azure DevOps/GitHub Actions/Jenkins).
• Apply data quality checks (Great Expectations or custom validations), monitoring, drift detection, and alerting.
• Model data for analytics (star/dimensional); publish to Synapse/Snowflake/SQL Server.
• Uphold governance and security (Purview/Unity Catalog lineage, RBAC, tagging, encryption, PII handling).
• Author documentation/runbooks; support production incidents and root-cause analysis; suggest cost/performance improvements.

Skills

Azure Data Factory

Databricks PySpark

Delta Lake

SQL & Python

Git CI/CD

Data modeling

Purview governance

RBAC governance

Kubernetes

Education

Bachelor’s/Master’s in CS/IT/Engineering

Tools

Azure Databricks

Synapse

Snowflake

Power BI

GitHub Actions

Jenkins

Purview

Unity Catalog

Job description

Senior Data Engineer Position: Senior Data Engineer Job Description: Job Title: Senior Data Engineer

Location : Pune/Ahmedabad/Indore

Role Summary

Build and operate scalable, reliable data pipelines on Azure. Develop batch and streaming ingestion, transform data using Databricks (PySpark/SQL), enforce data quality, and publish curated datasets for analytics and ML.

Key Responsibilities

Design, build, and maintain ETL/ELT pipelines in Azure Data Factory and Databricks across Bronze - Silver - Gold layers. Implement Delta Lake best practices (ACID, schema evolution, MERGE/upsert, time travel, Z-ORDER). Write performant PySpark and SQL; tune jobs (partitioning, caching, join strategies). Create reusable components; manage code in Git; contribute to CI/CD pipelines (Azure DevOps/GitHub Actions/Jenkins). Apply data quality checks (Great Expectations or custom validations), monitoring, drift detection, and alerting. Model data for analytics (star/dimensional); publish to Synapse/Snowflake/SQL Server. Uphold governance and security (Purview/Unity Catalog lineage, RBAC, tagging, encryption, PII handling). Author documentation/runbooks; support production incidents and root-cause analysis; suggest cost/performance improvements.

Must-Have (Mandatory)

• Data Engineering & Pipelines Hands-on experience building production pipelines with Azure Data Factory and Databricks (PySpark/SQL).
• Working knowledge of Medallion Architecture and Delta Lake (schema evolution, ACID).
• Programming & Automation Strong Python (pandas/PySpark) and SQL.
• Practical Git workflow; experience integrating pipelines into CI/CD (Azure DevOps/GitHub Actions/Jenkins).
• Familiarity with packaging reusable code (e.g., Python wheels) and configuration-driven jobs.
• Data Modeling & Warehousing Solid grasp of dimensional modeling/star schemas; experience with Synapse, Snowflake, or SQL Server.
• Data Quality & Monitoring Implemented validation checks and alerts; exposure to drift detection and pipeline observability.
• Cloud Platforms (Azure preferred) ADLS Gen2, Key Vault, Databricks, ADF basics (linked services, datasets, triggers), environment promotion.
• Data Governance & Security Experience with metadata/lineage (Purview/Unity Catalog), RBAC, secrets management, and secure data sharing.
• Understanding of PII/PHI handling and encryption at rest/in transit.
• Collaboration Clear communication, documentation discipline, Agile ways of working, and code reviews.
• Databricks Asset Bundles (DAB) for environment promotion/infra-as-code style deployments.
• Streaming/real-time: Kafka/Event Hubs; CDC tools (e.g., Debezium, ADF/Synapse CDC).
• MLOps touchpoints: MLflow tracking/registry, feature tables, basic model-inference pipelines.
• Power BI exposure for publishing curated tables and building operational KPIs.
• DataOps practices: automated testing, data contracts, lineage-aware deployments, cost optimization on Azure.

Microsoft Certified — Azure Data Engineer Associate (DP-203) or equivalent.

5+ years of professional experience in data engineering (or equivalent project depth).

Bachelor’s/Master’s in CS/IT/Engineering or related field (or equivalent practical experience).

Location: IN-GJ-Ahmedabad, India-Ognaj (eInfochips) Time Type: Full time Job Category: Engineering Services Experience Level Senior Level