Data & AI Platform Engineer

The Reliable Jobs; India; Remote; INR 1,500,000 - 2,100,000; Full time

Job summary

The Reliable Jobs is seeking a Data & AI Platform Engineer (remote, full-time) with 4+ years of experience. You will design and operate reliable data pipelines (batch, streaming, event-driven) using Python, SQL and Spark, build enterprise connectors, and contribute to RAG, memory and AI policy initiatives.

The role emphasizes cloud data services, Kubernetes operations, and scalable platform delivery. Responsibilities cover governance, security, provenance, and cross-functional collaboration to

Qualifications

• 4+ years of experience in data engineering or platform engineering.
• Experience designing and building batch, streaming and event-driven data pipelines.

Responsibilities

• Design, build and operate reliable data pipelines using Python, SQL and Spark.
• Implement governed storage, schema evolution and data-product patterns for structured/unstructured data.
• Build reusable components that support multiple products and tenants.
• Develop enterprise connectors for APIs, files, databases and event streams with secure access.
• Contribute to RAG, knowledge and memory services with provenance and citations.
• Support AI policy, quality engineering and measurable governance across data/AI products.

Skills

Python

SQL

Spark / PySpark

Cloud data services

Kubernetes

Tools

Kubernetes

Helm

GitOps

CI/CD

Job description

Job Description – Data & AI Platform Engineer

Location: Remote (Full time)

Experience: 4+ years

The client is building enterprise AI products and shared platform capabilities that combine modern data engineering with AI, cloud and platform engineering. The Data & AI Platform Engineering job family builds the dependable data and intelligence services used across products, client deployments and Catalyst platform capabilities.

Responsibilities

The balance of work will depend on the assigned product, project or Catalyst capability. Each engineer will contribute across the common foundation and lead or specialize in one or more areas below.Responsibilities

• Design, build and operate reliable batch, streaming and event-driven data pipelines using Python, SQL, Spark or PySpark and fit-for-purpose cloud data services.
• Implement governed storage, schema evolution, transformation, metadata and data-product patterns for structured and unstructured data.
• Build reusable components that support multiple products and tenants rather than creating one-off pipelines for each delivery.

2) Enterprise Connectors & Governed Access (20%)

• Build and maintain connectors for enterprise systems, APIs, files, databases, event streams and external data providers.
• Implement secure authentication, credential handling, identity propagation, tenant isolation and access controls for connector use.
• Expose consistent application and agent interfaces through APIs and, where appropriate, MCP tools with clear health, lifecycle and audit controls.

3) RAG, Enterprise Knowledge & Memory (20%)

• Engineer ingestion, extraction, chunking, enrichment, embedding, indexing and retrieval flows for enterprise RAG and knowledge services.
• Implement retrieval metadata, provenance, lineage, citations, permissions and quality signals from source content through to application responses.
• Contribute to governed Memory services that capture, qualify, retrieve and remove user or application context across sessions and agents.

4) Evaluation, AI Policy & Quality Engineering (15%)

• Build governed datasets, evaluators, quality checks, regression runs and release evidence for data, retrieval and AI capabilities.
• Implement measurable controls for accuracy, safety, policy compliance, traceability and production readiness.
• Support AI Policy and business-rule enforcement through versioned, testable and explainable service contracts.

5) Cloud Platform, Kubernetes & Operations (15%)

• Package and operate services using containers, Kubernetes, Helm, GitOps and CI/CD across local, SaaS and customer-managed deployment profiles.
• Implement configuration, secrets, identity, tenancy, scaling, resilience, backup and recovery controls appropriate to the service.
• Instrument services with logs, metrics and traces, and use operational evidence to improve performance, reliability and cost.

6) Technical Delivery & Engineering Leadership (10%)

• Translate product and architecture requirements into small, testable and supportable increments with clear acceptance criteria.
• Contribute to architecture decisions, technical design documents, code reviews, testing standards, runbooks and onboarding guidance.
• Work across product, data, AI, platform, security and operations teams to resolve dependencies and deliver reusable outcomes.

Specialization Areas

Data Integration & Enterprise Connectors: APIs, event-driven integration, enterprise systems, credential flows, identity-aware access and connector lifecycle engineering.

RAG & Enterprise Knowledge: document processing, embeddings, vector and keyword search, retrieval quality, metadata, lineage and citations.

Memory, Evaluation, AI Policy & Ontology: context capture and retrieval, governed evaluation, quality gates, policy enforcement, business meaning and explainable outcomes.

Cloud Data & Platform Engineering: cloud data services, Kubernetes, GitOps, platform APIs, security, tenancy, observability, reliability and deployment automation.