Senior Data Engineer [T500-29419]

Guidant Global; Hyderabad; Hybrid; INR 3,000,000 - 5,400,000; Full time

Benefits offered by this job

Founding India GCC

Global platform exposure

AI-enabled data engineering

Cross-border collaboration

Hyderabad role

Job summary

Guidant Global India seeks a Senior Data Engineer to own end-to-end Databricks Lakehouse pipelines, ensuring governance and scalable data products. You will work with UK, NL, and India teams to set engineering standards while delivering AI-ready data assets.

Be a founding member of the Hyderabad GCC, collaborating across time zones with a focus on reliability, security, and architectural excellence.

Qualifications

• 3+ years of production Azure Databricks experience with PySpark and Delta Lake.
• Medallion architecture design across Bronze/Silver/Gold with quality gates.
• Unity Catalog governance including RBAC and lineage.
• CI/CD deployment of Databricks workloads (DABs).
• Strong Python and Spark SQL for transformations.
• Data contracts and schema versioning in a multi-consumer Lakehouse.
• Databricks Certification or active pursuit; English fluency.

Responsibilities

• Design, build, and own end-to-end Databricks Lakehouse pipelines from ingestion to data products.
• Maintain Unity Catalog governance, security, and data contracts.
• Deploy pipelines via DABs and CI/CD across dev, test, prod.
• Adhere to FITT principles and contribute to platform standards.
• Mentor medior engineers and collaborate with global teams.

Skills

Azure Databricks

PySpark

Delta Lake

Unity Catalog governance

DABs CI/CD

Python

Spark SQL

Data contracts

English communication

Tools

SnapLogic

Azure DevOps

GitHub Actions

Azure Synapse

Job description

Guidant Global delivers dynamic, tailored workforce solutions that empower businesses to thrive in ever-evolving markets. Through our MSP, RPO, Direct Sourcing, Services Procurement and Consulting services, we help organisations find, engage, and manage the best permanent and contingent talent across the globe.

Part of Impellam Group, we champion A Better Way, a people-first approach that integrates deep industry expertise with advanced technology. Our technology turns workforce complexity into clarity, with real-time insights that support smarter talent decisions at scale.

With over 1.5 million STEM and professional experts in our network across 80+ countries, we don't just solve today's challenges. We're helping to shape the workforce of tomorrow.

About Guidant Global India:

At Guidant Global India, we play our part in shaping the future of work by powering recruitment and support services for Guidant teams across APAC and wider global markets. As a strategic delivery hub and global capability centre, we partner with stakeholders worldwide to raise quality, strengthen compliance, and deliver consistently at scale.

We're not standing still. We're building deeper capability in-house, taking ownership of the work that matters most, and designing an agile, future-ready delivery model built for where the business is heading next.

Join us, and you'll be part of a people-first team that's expert, ambitious, and collaborative, delivering A Better Way, every day.

Role Overview:

HFG is building its Hyderabad Global Capability Centre (GCC) as the long-term engineering backbone of its global data platform. This is a permanent, full-time role requiring 5+ years' total experience with 3+ years in production Databricks, working a minimum four-hour overlap with either CET (approx. 09:00–13:00 IST) or US Eastern/Central hours (approx. 18:00–22:00 IST) across a 40 hour week. We are looking for Senior Data Engineers who will be founding members of this team — engineers who own the full depth of the platform, from Lakeflow ingestion through medallion pipelines to AI-ready data products, and who will help set the engineering standards the GCC will run on for years to come. Working closely with the Global Head of Data & AI and senior engineers in the UK and Netherlands, the Senior Data Engineer helps set the technical standards and mentoring culture the India GCC engineering team will run on as it scales.

About HFG & the Data Platform:

Headfirst Global (HFG) is a global staffing and workforce solutions company operating across the EU, UK, North America, and APAC. The Global Data & AI team is building the Headless Data Architecture (HDA) — a hub-and-spoke federated lakehouse platform on Azure Databricks, enabling analytics, AI, and operational intelligence across the group's global business units. The platform runs on Azure Databricks with Unity Catalog, Lakeflow, Mosaic AI, and AgentBricks as core components, and integrates with SnapLogic for enterprise connectivity. All AI workloads route through the Unity AI Gateway. The India GCC team is being built as the long-term engineering backbone of this platform, working in close collaboration with senior engineers in the UK, Netherlands, and the US.

Key Responsibilities:

Design, build, and own end-to-end Databricks Lakehouse pipelines — from Lake flow ingestion through Bronze/Silver/Gold medallion layers to Gold-layer data products consumed by analytics and AI workloads.

• Implement and maintain Unity Catalog governance — catalog and schema design, RBAC, column-level security, lineage, and data contracts.
• Deploy and manage pipelines using Databricks Asset Bundles (DABs) and Azure DevOps/GitHub Actions CI/CD across dev, test, and production environments.
• Design reusable, idempotent, and testable pipelines adhering to HFG FITT standards (Functional, Idempotent, Tested, Two-stage).
• Lead technical design discussions and contribute architectural decisions to the India GCC platform standards.
• Mentor and support medior data engineers on the team, reviewing code and establishing engineering best practices.
• Integrate enterprise source systems through SnapLogic, ADF, and Databricks-native connectors including Salesforce, Dynamics 365, and SAP.
• Contribute to AI-enabled data engineering — building AI-ready Delta Lake datasets, vector search pipelines, and LLM-adjacent data products.
• Own production reliability — monitoring, alerting, Delta table maintenance (OPTIMIZE, VACUUM, liquid clustering), and incident response.
• Collaborate daily with UK and NL senior engineers in code reviews, architecture reviews, and sprint planning.

Qualifications & Experience:

Must-have:

• 3+ years of production Azure Databricks experience — PySpark, Delta Lake, Databricks Workflows.
• Proven medallion architecture design across Bronze, Silver, and Gold layers with explicit quality gates.
• Unity Catalog governance — RBAC, fine-grained access control, lineage.
• Databricks Asset Bundles (DABs) or equivalent CI/CD deployment of Databricks workloads.
• Strong Python and Spark SQL for complex transformation logic.
• Data contracts and schema versioning practices in a multi-consumer Lakehouse environment.
• Databricks Certified Data Engineer Associate or Professional certification (or active pursuit).
• SnapLogic or comparable iPaaS tool (MuleSoft, Boomi) — building and maintaining enterprise integration flows.
• Azure Synapse Analytics — familiarity with Dedicated SQL Pool and Synapse pipelines for maintaining existing flows during the Databricks migration.
• Excellent written and spoken English — comfortable in daily international collaboration.

Nice to have:

• Delta Live Tables (DLT) or Lake flow Connect for streaming ingestion.
• Snap Logic or comparable iPaaS integration experience.
• AI-enabled DE work: RAG pipelines, Databricks Vector Search, MLflow, LangChain/LangGraph.
• Experience working in staffing, HR-tech, or workforce intelligence platforms.

This role is based in Hyderabad and works in a hybrid model. Daily collaboration with HFG teams requires a minimum four-hour overlap with either CET (approx. 09:00–13:00 IST) or US Eastern/Central hours (approx. 18:00–22:00 IST) across a 40 hour week. Candidates should be comfortable with structured asynchronous communication for the remaining hours. The India GCC team meets daily with the global Data & AI team for standups, code reviews, and delivery planning. Strong written and spoken English communication is essential.

Our Platform Stack:

Azure Databricks — Delta Lake, Unity Catalog, DLT, DABs, Lake flow, Mosaic AI

• SnapLogic — enterprise integration and data orchestration
• Python, PySpark, Spark SQL
• Unity AI Gateway — governed LLM routing and observability
• MLflow, Databricks Vector Search, AgentBricks.

What We Offer:

• Be a founding member of the India GCC — shape how the team is built and the engineering standards it runs on
• Work on a globally scaled platform serving EU, UK, US, and APAC business units simultaneously
• Direct exposure to AI-enabled data engineering on one of the most modern Databricks deployments in the staffing industry
• Collaborate with senior engineers across the UK and Netherlands who will actively invest in your growth
• Hyderabad-based with a global engineering mandate — not an offshore support function.