Data Platform Engineer III

The Standard India; Bengaluru; Hybrid; INR 2,800,000 - 4,200,000; Full time

Benefits offered by this job

Annual incentive bonus

Paid time off

Career advancement opportunities

Learning and leadership enablement

Supportive management

Job summary

The Standard India is seeking a Data Platform Engineer III to own ingestion pipelines, governance and data movement on the Data & AI Platform. You will work with Databricks, Neo4j, Lakebase, Confluent Kafka and Unity Catalog, ensuring freshness, reliability and contract adherence across the enterprise data lake.

Hybrid role based in Bengaluru with on-site work for key engagements; remote flexibility available as needed.

Qualifications

• 4+ years of data engineering experience building production pipelines.
• Strong PySpark and SQL skills and solid Python fundamentals.
• Hands‑on Databricks experience: Delta Lake, jobs/workflows and Unity Catalog.
• Kafka/Confluent experience, including consuming event streams into analytical stores.
• Experience with pipeline orchestration (Lakeflow/Delta Live Tables, Airflow, Azure Data Factory or equivalent).
• Experience processing unstructured data (documents, text) in pipelines.
• Working knowledge of a graph or vector data store (Neo4j, Databricks Vector Search, Azure AI Search or equivalent).
• Data-quality testing experience (Great Expectations, dbt tests or equivalent) and lineage/freshness monitoring.

Responsibilities

• Build and operate batch and streaming ingestion pipelines on Databricks (PySpark, Lakeflow/Delta Live Tables).
• Integrate event streams from Confluent Kafka into the Lakehouse and manage schema evolution.
• Build ingestion and parsing pipelines for structured and unstructured content feeding the lakehouse, graph and vector stores.
• Operate and maintain Neo4j environments, backups/restores, monitoring and routine upgrades.
• Execute disaster recovery procedures and multi-region runbook execution.
• Automate data-quality and contract testing in CI/CD and respond to pipeline incidents.

Skills

PySpark

SQL

Python

Databricks

Unity Catalog

Kafka/Confluent

Airflow

Azure Data Factory

Delta Live Tables

Neo4j

Databricks Vector Search

Azure AI Search

Great Expectations

dbt tests

Git/CI/CD

Education

B.Tech / B.E. in Software Engineering, Computer Science or Information Systems

Tools

Lakeflow

Delta Lake

Unity Catalog

Job description

The next part of your journey is right around the corner — with StanCorp Global Services. A genuine desire to make a difference in the lives of others is the foundation for everything we do. With a customer-first mindset and an intentional focus on building strong teams across the nation, we’ve been able to uphold our legacy of financial stability while investing in new, innovative technologies that support the needs of our customers. Our high-performance culture, focused on operational excellence, thrives thanks to remarkable people united by compassion and a customer-first commitment. Are you ready to make a difference?

Job Summary

The Standard India is seeking a Data Platform Engineer III for pipeline and platform data engineering on our Data & AI Platform: Databricks, Lakebase, Confluent Kafka, ingestion pipelines, lineage, freshness and data movement. In this role, you will keep governed data products fresh, tested and within contract. You will also support day-to-day operations of the platform's graph and vector stores — Neo4j, Databricks Vector Search and Azure AI Search — under runbooks established by the Data Platform Engineer IV, including backup/restore and disaster recovery drills. Your initial assignment makes you the primary owner for Lakebase, ingestion, lineage and freshness handling on the Enterprise Memory task force, with supporting work on Databricks and data platform connectivity for the MCP Gateway task force. This role sits on The Standard's Data & AI Platform team, which owns the enterprise's shared data and AI foundation — governed once and consumed by many — built on Azure Databricks, Neo4j, Confluent Kafka, Kong, and Anthropic and OpenAI models behind a governed gateway.

• Ability to work on-site in Bengaluru, India is a requirement of the role. *

Mode of Working: This role follows a hybrid work model, with a primary base in Bengaluru. While on-site presence is essential for key engagements, flexibility is offered for remote work based on business needs and team alignment.

Key Responsibilities

• Build and operate batch and streaming ingestion pipelines on Databricks (PySpark, Lakeflow/Delta Live Tables).
• Integrate event streams from Confluent Kafka into the Lakehouse and manage schema evolution.
• Build and operate Lakebase-backed operational data patterns alongside the lakehouse.
• Build ingestion and parsing pipelines for structured and unstructured content (documents, text) feeding the lakehouse, graph and vector stores.
• Operate and maintain Neo4j environments under established runbooks: backups and restores, monitoring, health checks and routine upgrades.
• Support vector index operations in Databricks Vector Search and Azure AI Search: run index refresh and sync pipelines and monitor retrieval data freshness.
• Execute disaster recovery procedures for platform data stores: backup/restore testing, failover drills and multi-region runbook execution.
• Implement lineage capture and freshness monitoring against published data contracts and SLAs.
• Automate data-quality and contract testing in CI/CD so violations fail fast, and respond to pipeline incidents.
• Maintain catalog and metadata hygiene in Unity Catalog, and contribute runbooks and operational documentation for the pipelines you own.

Skills And Background You’ll Need

Education: B.Tech / B.E. in Software Engineering, Computer Science or Information Systems is required.

Experience

• Typically requires 4+ years of data engineering experience building production pipelines.
• Strong PySpark and SQL skills and solid Python fundamentals.
• Hands‑on Databricks experience: Delta Lake, jobs/workflows and familiarity with Unity Catalog.
• Kafka/Confluent experience, including consuming event streams into analytical stores.
• Experience with pipeline orchestration (Lakeflow/Delta Live Tables, Airflow, Azure Data Factory or equivalent).
• Experience processing unstructured data (documents, text) in pipelines.
• Working knowledge of a graph or vector data store (Neo4j, Databricks Vector Search, Azure AI Search or equivalent), including routine operations such as backups, monitoring and index refreshes.
• Data-quality testing experience (Great Expectations, dbt tests or equivalent) and lineage/freshness monitoring.
• Git‑based workflows and CI/CD for data pipelines

Preferred Experience

• Cypher basics and interest in growing into graph engineering.
• Disaster recovery or multi‑region operations exposure (replication, failover drills, backup/restore testing).
• Lakebase or Postgres‑style operational store experience.
• Confluent ecosystem depth (Schema Registry, connectors) and CDC tooling.
• Observability tooling (Datadog) and on‑call experience.
• Good to have Databricks Data Engineer Associate or Confluent Certified Developer.

Key Behaviors Of a Successful Candidate

• Adaptability — Seeks information to understand the rationale for and importance of the change.
• Customer Focus — Displays an interest in the customer by trying to understand their concerns and issues; draws on customer insight to help others best meet current and future customer needs.
• Improvement Mindset — Demonstrates curiosity by asking questions regarding current approaches/methods and identifying potential changes.

Why Join Standard India?

• A rich benefits package that supports your health, well‑being, and financial goals
• An annual incentive bonus plan tied to both individual and organizational performance
• Paid time off including earned leave, sick/casual leave, and public holidays, in accordance with the company’s leave policy
• A supportive, responsive management approach that empowers you to grow
• Opportunities for career advancement through learning, coaching, and leadership enablement
• A culture where your voice matters—your ideas shape our future, and we win as one
• Purpose‑driven work that makes a difference for our customers, communities, and each other

In addition to a competitive salary, our employee‑focused benefits are designed to support work‑life balance, personal growth, and long‑term impact.

Eligibility to participate in an incentive program is subject to the rules governing the program and plan. Any award depends on a variety of factors including individual and organizational performance.

StanCorp Global Services India Private Limited is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, religion, color, sex, gender identity, sexual orientation, age, disability, caste, HIV status or any other characteristic protected by applicable laws of India.