Senior Data Platform Engineer

Mirantis; India; Remote; INR 4,000,000 - 7,000,000; Full time

Benefits offered by this job

Competitive compensation package with强

Professional development and training

Attend conferences and working groups

Company outings, hackathons, and tech

Job summary

Mirantis, an IREN company, seeks an experienced data platform engineer to own the event-streaming backbone for the k0rdent-ai platform. You will design, deploy, and operate the streaming platform on Kubernetes, including topics, partitioning, retention, and upgrades, while building CDC pipelines and schema governance to keep producers and consumers in sync.

Join a global engineering team across the US, Europe, and India to deliver reliable data movement to analytics, billing, observability, and

Qualifications

• 10+ years in software, data, or platform engineering with production event-streaming ownership.
• Expert-level Kafka administration, tuning, and capacity management at production scale.
• Strong Kafka Connect and CDC experience with Debezium or equivalents.
• Schema registry and compatibility management across producers/consumers.
• Proficient in Java/Python/Go to build connectors and tooling.
• Strong SQL/relational DB knowledge (PostgreSQL/MySQL).
• Kubernetes fluency for stateful workloads.
• Infrastructure as code (Terraform) and CI/CD ownership.
• Clear written English for design docs and runbooks.

Responsibilities

• Own production streaming platform on Kubernetes including topics, partitions, and retention.
• Build and maintain CDC and connector pipelines to analytics stores.
• Manage schema governance and compatibility across producers/consumers.
• Develop connectors and stream-processing logic in Java or Go.
• Create self-service onboarding for topics, schemas, and access via IaC.
• Automate platform with Terraform and GitOps; ensure proper topic definitions.
• Instrument pipeline with metrics, monitoring, and dashboards for teams.
• Ensure multi-tenant security: mTLS, RBAC, encryption, audit trails.
• Collaborate with backend/SRE on event schema and failure modes; participate in on-call.
• Mentor engineers on streaming architecture and raise technical bar.

Skills

Apache Kafka

Kafka Connect

Go

Java

Python

SQL

Kubernetes

Terraform

CI/CD

English communication

Education

Bachelor’s degree in Computer Science or related field

10 years related experience

Tools

Debezium

Schema Registry

Confluent Platform

JDBC

Kafka Connect

Terraform

GitHub Actions

Jenkins

Kubernetes

Job description

Mirantis, an IREN company, is the Kubernetes-native AI infrastructure company, enabling organizations to build and operate scalable, secure, and sovereign infrastructure for modern AI, machine learning, and data-intensive applications. By combining open source innovation with deep expertise in Kubernetes orchestration, Mirantis empowers platform engineering teams to deliver composable, production-ready developer platforms across any environment—on-premises, in the cloud, at the edge, or in sovereign data centers. As enterprises navigate the growing complexity of AI-driven workloads, Mirantis delivers the automation, GPU orchestration, and policy-driven control needed to manage infrastructure with confidence and agility. Committed to open standards and freedom from lock-in, Mirantis ensures that customers retain full control of their infrastructure strategy. https://www.mirantis.com/

Job Description

We are looking for an experienced data platform engineer to own the event-streaming backbone behind the k0rdent-ai platform — the multi-tenant control plane for enterprise GPU infrastructure. Every cluster provisioned, every GPU-hour consumed, and every tenant action produces events that must move reliably from our services into analytics, billing, observability, and downstream systems. You will design and operate that pipeline end to end: the streaming platform itself, the connectors and change-data-capture flows that feed it, the schemas that keep producers and consumers compatible, and the automation that lets product teams onboard themselves without waiting on you. Working within an agile framework alongside engineering teams across the US, Europe, and India, you will make event data a dependable product surface rather than a best-effort side channel.

Main Responsibilities:

Design, deploy, and operate the production streaming platform on Kubernetes — cluster lifecycle, topics, partitioning and retention strategy, capacity planning, and upgrades.

Build and maintain change-data-capture and connector pipelines moving data between operational databases, the streaming layer, and analytical stores.

Own schema governance — a schema registry, compatibility rules, and versioning discipline so producer changes never silently break consumers.

Develop custom connectors, transformations, and stream-processing logic in Java or Go where off-the-shelf components fall short.

Build self-service onboarding so product teams can provision topics, schemas, and access through infrastructure as code rather than tickets.

Automate the platform with Terraform and GitOps — no manually configured clusters, no undocumented topic definitions.

Instrument the pipeline for production: metrics, lag and throughput monitoring, alerting, and dashboards that both the platform team and application owners can act on.

Enforce security and multi-tenant isolation across the streaming layer — mTLS, authentication, topic-level RBAC, encryption, and audit trails.

Guarantee delivery semantics that the business can rely on — ordering, idempotency, exactly‑once where required, replay, and disaster recovery.

Partner with backend and SRE teams on event schema design, producer patterns, and failure modes; participate in on‑call for the platform you own.

Mentor engineers on streaming architecture and raise the team's bar through design review and documentation.

Qualifications

Required Skills/Abilities:

10+ years in software, data, or platform engineering, including deep hands‑on ownership of a production event‑streaming platform.

Expert-level Apache Kafka (or Confluent Platform/Cloud) — administration, tuning, troubleshooting, and capacity management at production scale.

Strong Kafka Connect and CDC experience — Debezium, JDBC, and custom connector or transformation development.

Proven schema‑registry practice and compatibility management across many independent producers and consumers.

Solid programming ability in Java, Python, or Go — enough to build connectors, transformations, and platform tooling, not only configure them.

Strong SQL and relational database knowledge (PostgreSQL, MySQL, or equivalent), including replication and CDC mechanics.

Kubernetes fluency — deploying, operating, and debugging stateful workloads.

Infrastructure as code discipline (Terraform or equivalent) and CI/CD pipeline ownership.

Clear written English for design docs, runbooks, and asynchronous review across global time zones.

Must Have

Streaming: Apache Kafka or Confluent, Kafka Connect, Schema Registry, and stream processing.

Data Movement: CDC tooling (Debezium or equivalent), connector development, and pipelines into analytical stores.

Kubernetes Native: Kubernetes, Docker, and Helm‑packaged workloads in production.

Automation: Terraform, CI/CD (Jenkins, GitHub Actions, or equivalent), and GitOps practice.

Data Stores: PostgreSQL at scale, plus at least one cloud data warehouse or search store (Snowflake, BigQuery, Elasticsearch, or equivalent).

Cloud: AWS or GCP — networking, IAM, managed Kubernetes.

Security & Observability: mTLS, OIDC/SSO, RBAC, and Prometheus/Grafana monitoring.

Nice to Have

Confluent certification, or contributions to Kafka‑ecosystem open source.

Stream processing with Flink, Kafka Streams, or ksqlDB.

Exposure to GPU infrastructure, AI/ML data pipelines, or telemetry at high cardinality.

Usage metering, billing, or chargeback pipelines built on event data.

Alternative brokers (NATS JetStream, Pulsar) and a view on the tradeoffs.

Go, for working directly in our backend services' producer code.

Data quality, lineage, or catalog tooling.

Compliance exposure — SOC 2, ISO 27001, or similar audit support.

Education and Experience:

Bachelor’s degree in Computer Science & Engineering or related field or 10 years related experience.

Additional Information

What does Mirantis offer you?

• Work with an established Silicon Valley leader in the cloud infrastructure industry;
• Work with exceptionally passionate, talented and engaging colleagues, helping Fortune 500 and Global 2000 customers implement next‑generation cloud technologies;
• Be a part of cutting‑edge, open‑source innovation;
• Thrive in the high‑energy environment of a young company where openness, collaboration, risk‑taking, and continuous growth are valued;
• Professional development and training;
• Attend conferences and working groups;
• Company outings, happy hours, hackathons, and tech talks;
• Receive a competitive compensation package with a strong benefits plan.