Databricks - Senior Engineer

Iris Software Inc.; Dadri; On-site; INR 1,000,000 - 1,500,000; Full time

Job summary

Iris Software invites a data engineer to join its expert team in India, Dadri. The role focuses on building scalable data pipelines, lakehouse architectures, and governance across analytics platforms.

You will work with PySpark, Kafka, and Databricks to deliver enterprise-grade data solutions, mentored by senior engineers and exposed to cutting-edge technologies in a fast-growing IT services environment.

Qualifications

• Mandatory skills include PySpark, Apache Kafka, Databricks Workflows and Delta Lake on Databricks.
• Experience in designing scalable data pipelines and modern data platforms is expected.
• Familiarity with data lakehouse architectures and governance practices is preferred.

Responsibilities

• Design scalable data engineering solutions using PySpark and distributed data processing.
• Define ingestion, transformation, and processing architectures to meet analytics needs.
• Design and optimize Snowflake or Delta Lake on Databricks solutions for enterprise-scale platforms.
• Lead high-performance batch and streaming data pipelines.
• Architect event-driven data systems using Kafka or Kinesis.
• Define streaming standards, integration frameworks, and processing patterns.
• Create workflow orchestration using Apache Airflow or Databricks Workflows.
• Establish monitoring, scheduling, and controls for reliable pipelines.
• Drive data quality, validation, reconciliation, and governance across solutions.
• Design with Lakehouse principles to improve scalability, reliability, and observability.
• Develop business-focused data products with quality, discoverability, and usability.
• Promote AI-assisted engineering to boost productivity and quality.
• Review designs to ensure scalability and performance, and troubleshoot issues.
• Mentor team members on the tools and best practices in data engineering.
• Collaborate with teams to support end-to-end data platform delivery.

Skills

PySpark

Apache Kafka

Databricks Workflows

Delta Lake on Databricks

Job description

Why Join Iris?

Are you ready to do the best work of your career at one of India’s Top 25 Best Workplaces in IT industry? Do you want to grow in an award-winning culture that truly values your talent and ambitions?

Join Iris Software — one of the fastest-growing IT services companies — where you own and shape your success story.

About Us

At Iris Software, our vision is to be our client’s most trusted technology partner, and the first choice for the industry’s top professionals to realize their full potential.

With over 4,300 associates across India, U.S.A, and Canada, we help our enterprise clients thrive with technology‑enabled transformation across financial services, healthcare, transportation & logistics, and professional services.

Our work covers complex, mission‑critical applications with the latest technologies, such as high‑value complex Application & Product Engineering, Data & Analytics, Cloud, DevOps, Data & MLOps, Quality Engineering, and Business Automation.

Working with Us

At Iris, every role is more than a job — it’s a launchpad for growth.

Our Employee Value Proposition, "Build Your Future. Own Your Journey." reflects our belief that people thrive when they have ownership of their career and the right opportunities to shape it.

We foster a culture where your potential is valued, your voice matters, and your work creates real impact. With cutting‑edge projects, personalized career development, continuous learning and mentorship, we support you to grow and become your best — both personally and professionally.

Curious what it’s like to work at Iris? Head to this video for an inside look at the people, the passion, and the possibilities. Watch it here.

Job Description

Mandatory Skills

PySpark, Apache Kafka, Databricks Workflows, Delta Lake on Databricks

Key Responsibilities

• Design scalable data engineering solutions using PySpark and modern distributed data processing frameworks.
• Define data ingestion, transformation, and processing architectures aligned with business and analytical objectives.
• Design and optimize Snowflake or Delta Lake on Databricks solutions to support enterprise‑scale data platforms.
• Lead implementation of high‑performance batch and streaming data pipelines.
• Design and optimize event‑driven data architectures using Apache Kafka or Amazon Kinesis.
• Define data streaming standards, integration frameworks, and scalable processing patterns.
• Architect workflow orchestration solutions using Apache Airflow or Databricks Workflows.
• Establish monitoring, scheduling, and operational controls for reliable pipeline execution.
• Drive data quality, validation, reconciliation, and governance practices across data engineering solutions.
• Design data engineering solutions following modern Lakehouse architecture principles, data observability practices, and platform engineering standards to improve scalability, reliability, and operational visibility.
• Drive development of business‑focused data products by improving data quality, discoverability, usability, documentation, and trusted data consumption across analytical platforms.
• Promote responsible use of AI‑assisted engineering capabilities to improve development productivity, testing, documentation, and engineering quality.
• Review data pipeline designs and implementations to ensure adherence to engineering, scalability, and performance standards.
• Troubleshoot complex data processing, workflow, and streaming platform issues through detailed root cause analysis.
• Mentor team members on PySpark, Snowflake, Delta Lake, Kafka, Kinesis, Airflow, and data engineering best practices.
• Collaborate with various teams and stakeholders to support end‑to‑end data platform delivery.

Behavioral Competencies

• Demonstrates strong ownership while driving data engineering excellence.
• Collaborate effectively with various teams and business stakeholders to ensure smooth delivery.
• Promotes quality‑focused engineering through proactive validation, optimization, and continuous improvement.