Data Engineer

Straive; Maharashtra; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Straive in Pune District, Maharashtra, India, is seeking a senior data engineering lead to architect and maintain enterprise-grade data pipelines. This on-site role requires senior expertise in Python, PySpark, Databricks, Kafka, and ML/AI tooling, with a focus on governance and compliance.

The candidate will design CI/CD pipelines, deploy microservices on OpenShift/Kubernetes, and implement data federation using Lambda/Data Mesh and Starburst to support AI/ML workflows.

Qualifications

• 8+ years in large-scale application development with 5+ years in a Python/PySpark data engineering lead role.
• Bachelor's degree in CS/Engineering or related field; master's preferred.
• Databricks certified or equivalent cloud data engineering credentials are preferred.

Responsibilities

• Architect and maintain enterprise-grade ELT/ETL data pipelines using Python, PySpark, Kafka, and Databricks.
• Build and deploy GenAI agents with Google ADK, LLMs, and MCP in human-in-the-loop workflows.
• Design and deploy microservice integrations on OpenShift/Kubernetes with robust CI/CD.
• Implement data federation via Lambda/Data Mesh with Starburst for AI/ML use cases.
• Leverage Devin.AI and GitHub Copilot to accelerate engineering velocity while ensuring governance.

Education

Bachelor's degree in Computer Science, Engineering, or related field

Master's degree preferred

Tools

Python

PySpark

Databricks

Google ADK

LLMs

FastAPI

Spring Boot

Microservices

Kafka

SQL

Data Mesh

Starburst

Kubernetes

OpenShift

Docker

CI/CD Pipelines

Job description

About This Job

Straive

Location: Pune District, Maharashtra, India

Work Mode: On-site

Industry: Software Development,IT Services and IT Consulting

Job Description

Roles and Responsibilities

• Architect and maintain enterprise-grade ELT and ETL data pipelines using Python, PySpark, Kafka, and Databricks to manage large-scale risk data.
• Build and deploy GenAI agents utilizing Google ADK, Google Flash 2.5+ LLMs, and Model Context Protocol (MCP) integrated with Human-in-the-Loop workflows.
• Design, automate, and deploy microservice integrations for data-intensive applications on OpenShift and Kubernetes using robust CI/CD pipelines.
• Implement data federation layers supporting Lambda and Data Mesh architectures via Starburst to enable AI/ML and NLP use cases.
• Leverage agentic AI platforms and development assistants such as Devin.AI and GitHub Copilot with prompt engineering to increase engineering velocity.
• Enforce data governance, risk management policies, and regulatory compliance standard across all data platforms.

Preferred Candidate Profile

• Work Experience: 8+ years in large-scale application development with 5+ years in a Python and PySpark Data Engineering lead role.
• Educational Background: Bachelor's degree in Computer Science, Engineering, or a related field (Master's degree preferred).
• Core Technical Skills: Python, PySpark, Databricks, Google ADK, LLMs, FastAPI, Spring Boot, Microservices, Kafka, SQL, Data Mesh, Starburst.
• Infrastructure and Cloud: Kubernetes, OpenShift, Docker, Cloud-Native Infrastructure, CI/CD pipelines.
• Industry Context: Data engineering experience in Banking Risk, Retail Products, Cards, Mortgage, Deposits, or Wealth Management.
• Assumed Requirements / Certifications: Databricks Certified Data Engineer, AWS Certified Data Analytics, or Azure Data Engineer Associate.