Senior Data Engineer (APAC Region)

ANRGI TECH Pvt. Ltd.; Pune District; On-site; INR 1,600,000 - 2,000,000; Full time

Job summary

Anrgi Tech Private Limited in Pune, Maharashtra is seeking a skilled Data Engineer to design, build, and operate scalable data pipelines on the Databricks platform, with a focus on end-to-end workflows from ingestion to consumption.

The role requires strong PySpark, Python, SQL, and Delta Lake expertise, 5+ years in data engineering, and 2–3 years with Databricks; certifications are mandatory and you will mentor juniors.

Qualifications

• Minimum 5 years in data engineering or related roles.
• At least 2–3 years hands-on with Databricks platform.
• Must hold at least one data engineering certification.
• Experience building production data pipelines at scale.

Responsibilities

• Design, build, and operate scalable data pipelines on the Databricks platform.
• Develop end-to-end data workflows from ingestion through transformation to consumption.
• Implement robust error handling, monitoring, and alerting mechanisms.
• Ensure data pipeline reliability, performance, and maintainability.
• Optimize pipeline performance through efficient Spark job design and cluster configuration.
• Manage and orchestrate complex data workflows using Databricks Jobs and workflows.

Skills

PySpark

Databricks Platform

Delta Lake

Python

Spark SQL

SQL

Structured Streaming

DevOps / CI/CD

Git

Data Engineering principles

Tools

Databricks

Delta Lake

Git

Spark

Python

Job description

Full time | Anrgi Tech Private Limited | India

Posted On 13/08/2026

Job Information

Technology

INR 1600000 - 2000000

City Pune City

State/Province Maharashtra

411001

Job Description

Data Pipeline Development & Operations

• Design, build, and operate scalable and reliable data pipelines on the Databricks platform
• Develop end-to-end data workflows from ingestion through transformation to consumption
• Implement robust error handling, monitoring, and alerting mechanisms
• Ensure data pipeline reliability, performance, and maintainability
• Optimize pipeline performance through efficient Spark job design and cluster configuration
• Manage and orchestrate complex data workflows using Databricks Jobs and workflows

Legacy Code Modernization

• Refactor legacy code and data pipelines to PySpark for improved performance and scalability
• Migrate traditional ETL processes to modern ELT patterns on Databricks
• Assess existing codebases and identify opportunities for optimization and modernization
• Ensure backward compatibility and data integrity during migration processes
• Document refactoring approaches and create migration playbooks
• Collaborate with stakeholders to minimize disruption during code transitions
• Implement data quality checks and validation frameworks
• Design and maintain Delta Lake tables with appropriate optimization strategies
• Develop reusable code libraries and frameworks for common data engineering tasks
• Follow software engineering best practices including version control, testing, and CI/CD
• Participate in code reviews and provide constructive feedback to team members
• Troubleshoot and resolve data pipeline issues in production environments
• Work closely with data architects, analysts, and business stakeholders
• Collaborate with Infrastructure (Infra), Applications (Apps), and Cyber teams
• Share knowledge and best practices with Team *****
• Mentor junior data engineers on PySpark and Databricks technologies
• Document technical solutions and maintain comprehensive documentation

Essential Technical Skills

• Data Engineering: Strong foundation in data engineering principles, ETL/ELT processes, and data pipeline design patterns
• PySpark: Proven hands‑on experience developing data pipelines using PySpark, including DataFrames API, Spark SQL, and performance optimization
• Databricks Platform: Practical experience with Databricks workspace, cluster management, notebooks, and job orchestration
• Workspace AI Agent: Knowledge of Databricks Workspace AI Agent capabilities and integration
• Delta Lake: Understanding of Delta Lake features including ACID transactions, schema evolution, and optimization techniques
• Python: Strong Python programming skills for data processing and automation

Additional Technical Skills

• SQL proficiency for data querying and transformation
• Experience with cloud platforms (Azure, AWS, or GCP)
• Understanding of data governance and security best practices
• Knowledge of streaming data processing (Structured Streaming)
• Familiarity with DevOps practices and CI/CD pipelines
• Experience with version control systems (Git)
• Understanding of data quality frameworks and testing methodologies

Professional Experience

• Minimum 5 years in data engineering or related roles
• At least 2-3 years of hands‑on experience with Databricks platform
• Proven track record of refactoring legacy code to modern frameworks
• Experience building and maintaining production data pipelines at scale
• Background working across multiple data sources and formats

Required Certifications - mandatory to have at least one certification

Additional Certifications (Preferred)

• Databricks Certified Associate Developer for Apache Spark
• Cloud platform certifications (Azure Data Engineer Associate, AWS Certified Data Analytics, or Google Cloud Professional Data Engineer)
• Relevant data engineering or big data certifications

Soft Skills

• Strong problem-solving and analytical thinking abilities
• Excellent communication skills to explain technical concepts clearly
• Ability to work collaboratively in cross‑functional teams
• Self‑motivated with strong attention to detail
• Adaptable to changing priorities and technologies
• Client‑focused mindset with commitment to quality delivery

Anrgi Tech Private Limited is an Equal Opportunity Employer and does not discriminate on the basis of race or ethnicity, religion, sex, national origin, age, veteran disability or genetic information or any other reason prohibited by law in employment.