Lead Data Engineer

Kyndryl Inc.

Hyderabad

Hybrid

INR 1,500,000 - 2,100,000

Full time

4 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Kyndryl is seeking an AI Data Engineer to design high‑performance data infrastructure powering autonomous systems. You will build real‑time pipelines for RAG‑enabled reasoning over enterprise data, transforming unstructured IT logs into optimized vector embeddings.

You will ensure health of vector databases, develop semantic layers and knowledge graphs, and implement automated data guardrails to protect privacy and maintain data quality while advancing production pipelines and CI/CD for

Qualifications

  • 7+ years of experience in data engineering/data warehousing.
  • 4+ years of hands-on experience with Databricks/Spark and Snowflake.
  • Experience building metadata-driven ELT frameworks, reusable pipeline templates, data-quality frameworks.

Responsibilities

  • Architect for RAG: design and scale pipelines for Retrieval-Augmented Generation (RAG), transforming massive volumes of unstructured IT logs and documentation into optimized Vector Embeddings.
  • Scale vector infrastructure: maintain sub-second retrieval speeds for agentic reasoning loops in vector databases (e.g., Pinecone, Milvus, Weaviate).
  • Engineer semantic layers: build knowledge graphs and semantic layers for contextual AI reasoning in complex environments.
  • Automate data excellence: implement automated data guardrails to detect noise, bias, or PII before it reaches the model.
  • Solve meaningful challenges: bridge raw data sources and AI work; identify and resolve data quality issues at the source.
  • Progress to production: build, deploy, and maintain CI/CD pipelines for data infrastructure to keep context windows fresh and reliable.

Skills

Databricks/Spark
Snowflake
CI/CD
Data wrangling
Data quality
ETL/ELT
PII handling
Debugging

Tools

Databricks Workflows
Notebooks
Unity Catalog
Git
CI/CD

Job description

Partially Remote

Bangalore, Karnataka, India

Hyderabad, Telangana, India

Full time

R-71351

Who We Are

At Kyndryl, we run and reimagine the mission‑critical technology systems that drive advantage for the world’s leading businesses. We are at the heart of progress; with proven expertise and a continuous flow of AI‑powered insight, enabling smarter decisions, faster innovation, and a lasting competitive edge. For our people—Kyndryls—that means doing purposeful work that powers human progress. Join us and experience a flexible, supportive environment where your well‑being is prioritized and your potential can thrive.

The Role

As an AI Data Engineer at Kyndryl, you’ll be the architect behind the high‑performance data infrastructure that powers our autonomous systems. We aren't just moving tables; we’re building the real‑time pipelines that allow agentic AI to reason over the world’s most complex enterprise environments.

Your role

As an AI Data Engineer at Kyndryl, you’ll be the architect behind the high‑performance data infrastructure that powers our autonomous systems. We aren't just moving tables; we’re building the real‑time pipelines that allow agentic AI to reason over the world’s most complex enterprise environments.

What you will do
  • Architect for RAG: You’ll design and scale the pipelines for Retrieval‑Augmented Generation (RAG), transforming massive volumes of unstructured IT logs and documentation into optimized Vector Embeddings.
  • Scale vector infrastructure: You will be responsible for the health and performance of our vector databases (e.g., Pinecone, Milvus, or Weaviate), ensuring sub‑second retrieval speeds for agentic reasoning loops.
  • Engineer semantic layers: Move beyond simple ETL to build knowledge graphs and semantic layers that provide agents with the necessary context to navigate complex infrastructure puzzles.
  • Automate data excellence: Using a keen eye for detail, you’ll build automated data guardrails to detect noise, bias, or PII (Personally Identifiable Information) before it reaches the model, ensuring our AI remains safe and impactful.
  • Solve meaningful challenges: Serve as the bridge between raw, messy data sources and deep technical AI work, identifying and resolving quality issues at the source.
  • Progress to production: With a well‑defined methodology and software engineering prowess, you will build, deploy, and maintain the CI/CD pipelines for our data infrastructure, ensuring that our context window remains fresh and reliable.
Your Future at Kyndryl

You’ll be creating a well‑rounded skillset while gaining professional certifications and qualifications.You can evolve horizontally into other technical or non‑technical areas such as Technical Service Architect, or you can move vertically to become a Senior Project Manager or program manager. There are many different types and flavors of roles, so everyone can find the opportunity that suits them best.

Who You Are

You’re good at what you do and possess the required experience to prove it. However, equally as important – you have a growth mindset; keen to drive your own personal and professional development. You are customer‑focused – someone who prioritizes customer success in their work. And finally, you’re open and borderless – naturally inclusive in how you work with others.

Required Skills & Experience
  • 7+ years of experience in data engineering/data warehousing, 4+ years of hands‑on experience with Databricks/Spark and Snowflake.
  • Advanced hands‑on experience with: Databricks Python SQL Spark/PySpark Delta Lake Snowflake
  • Strong experience with Databricks Workflows/Jobs, notebooks, Unity Catalog, Git, and CI/CD.
  • Experience building metadata‑driven ELT frameworks, reusable pipeline templates, data‑quality frameworks, reconciliation utilities, and automated testing.
  • Strong knowledge of performance tuning, partitioning, incremental processing, schema evolution, and large‑volume data processing.
  • Knowledge of dimensional and normalized data modeling.
  • Experience working with healthcare data, including patient identity, effective dating, PHI handling, lineage, and audit requirements.
  • Strong troubleshooting, debugging, and root‑cause‑analysis skills.
Preferred Skills:
  • Experience building GenAI/RAG applications on Databricks, Agentic AI / AI Agents, Generative AI and LLMs, Retrieval‑Augmented Generation (RAG), Prompt Engineering, Embeddings & Vector Search
  • Experience with MLflow and AI/ML lifecycle practices.
  • Exposure to Azure OpenAI or equivalent enterprise LLM platforms.
  • Knowledge of agent frameworks/orchestration approaches.
  • Experience moving AI/GenAI solutions from POC to production, including CI/CD, monitoring, security, and governance.
  • Healthcare domain experience and familiarity with PHI‑sensitive datasets
Being You

The “Kyn” in Kyndryl means kinship, which represents the strong bonds we have with each other, our customers and our communities. We focus on ensuring all Kyndryls feel included and we welcome people of all cultures, backgrounds, and experiences. Even if you don’t meet every requirement, we encourage you to apply. We believe in growth, and we’re excited to see what you can bring. At Kyndryl, employee feedback has told us that our number one driver of employee engagement is belonging. That sense of belonging — being a valued, respected, trusted member of the team — is fundamental to our culture and fueling great experiences for our customers. This dedication to welcoming everyone into our company means that Kyndryl gives you the ability to thrive and contribute to our culture of empathy and shared success. That’s The Kyndryl Way.

What You Can Expect

Your career with us isn’t just a job—it’s an adventure with purpose. We offer a dynamic, hybrid‑friendly culture that supports your well‑being and empowers you to grow. Our Be Well programs are thoughtfully designed to support your financial, mental, physical, and social health—because we know that when you feel your best, you do your best.

From your very first day, you’ll dive into impactful work that powers the systems our customers rely on every day. You won’t just contribute—you’ll make a difference, tackling meaningful projects that sharpen your skills and fuel your growth.

We’re here to champion your journey. With powerful tools to chart your career path, personalized development goals aligned with your ambitions, and continuous feedback to keep you inspired and on track, you’ll have everything you need to thrive and evolve. You’ll develop in‑demand skills to grow your career and achieve your ambitions with access to cutting‑edge learning opportunities—from certifications with Microsoft, Google, and Amazon to coaching and hands‑on experiences. And through it all, you’ll be part of a culture that values empathy, restless learning, and a devotion to shared success.

We want you to thrive here—and we’re committed to helping you do just that. Ready to make an impact? Join us and help shape what’s next.

At Kyndryl, we achieve progress the world depends on, with purpose. Beginning with the purpose that matters to you. Because here, you will be part of a culture designed with purpose. One that is restless, empathetic and devoted. Where we are committed to sustainable progress for our customers and supporting the communities where we work and live. All of you is what we want. And what we need. Join us, and together, we can advance the vital systems that power human progress.

Kyndryl is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, pregnancy, disability, age, veteran status, or other characteristics. Kyndryl is also committed to compliance with all fair employment practices regarding citizenship and immigration status.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Data Engineer
Lead Data Engineer

Kyndryl Inc. • Bengaluru

Hybrid
INR 2,800,000 - 4,500,000
Data Operations Lead – Databricks, dbt & Azure
Data Operations Lead – Databricks, dbt & Azure

Kyndryl Inc. • Bengaluru

On-site
INR 4,000,000 - 7,000,000
Global Data Steward
Global Data Steward

Kyndryl Inc. • Bengaluru

On-site
INR 1,800,000 - 3,000,000
Data Engineer – Azure Data Platform
Data Engineer – Azure Data Platform

Kyndryl Inc. • Bengaluru

On-site
INR 1,800,000 - 3,600,000
Be Well program
Hybrid-friendly culture
Lead Data Engineer
Lead Data Engineer

5100 Kyndryl Solutions Private Limited • Karnataka

Hybrid
INR 1,800,000 - 2,800,000
Data Engineer – Databricks & dbt
Data Engineer – Databricks & dbt

Kyndryl Inc. • Bengaluru

On-site
INR 4,500,000 - 7,000,000
Data Analyst – Databricks & Power BI
Data Analyst – Databricks & Power BI

Kyndryl Inc. • Bengaluru

On-site
INR 900,000 - 1,500,000
Be Well program
Hybrid-friendly culture
Career growth opportunities
Senior Data Engineer – Databricks, dbt & GenAI
Senior Data Engineer – Databricks, dbt & GenAI

Kyndryl Inc. • Chennai

Hybrid
INR 3,000,000 - 6,000,000
Data Operations Lead – Databricks, dbt & Azure
Data Operations Lead – Databricks, dbt & Azure

5100 Kyndryl Solutions Private Limited • Chennai

Hybrid
INR 3,500,000 - 5,200,000
Data Scientist
Data Scientist

5100 Kyndryl Solutions Private Limited • Gurugram District

Hybrid
INR 2,200,000 - 4,000,000
Be Well program
Hybrid-friendly culture