Data Engineer â OTA & Telemetry Infrastructure

TopGrep Tech Private Limited; Bengaluru; On-site; INR 1,800,000 - 2,400,000; Full time

Job summary

TopGrep Tech Private Limited is seeking an experienced Data/IoT Backend Engineer to own OTA firmware lifecycle for multi-ECU vehicle architectures. You will design automated failure detection, retry, rollback, and ensure secure OTA pipelines while building real-time telemetry and data processing services in Rust on AWS.

You will work with Kafka, AWS IoT Core, MemoryDB, and Python/SQL for data processing, dashboards, and monitoring, delivering resilient pipelines across intermittent connectivity

Qualifications

• 4–6 years of experience in Data Engineering, Software Engineering, or IoT Backend Engineering.
• Strong hands-on experience with Rust for backend/data processing applications.
• Experience with AWS, particularly IoT Core, S3, ECS/EKS, and Lambda.
• Strong experience with Apache Kafka and real-time data pipelines.
• Hands-on experience with AWS MemoryDB for Redis or Redis Enterprise.
• Working knowledge of Python and SQL.
• Experience with MQTT, WebSockets, and HTTP/S protocols.
• Strong understanding of distributed systems, streaming data, and resilient data pipelines.

Responsibilities

• Co-own OTA firmware rollout operations across multi-ECU connected vehicle architectures.
• Design automated mechanisms to detect update failures, network interruptions, verification errors, and stalled deployments.
• Implement deterministic retry, recovery, and rollback mechanisms to ensure reliable firmware updates without vehicle bricking.
• Ensure firmware package integrity, signature validation, and data security throughout the OTA pipeline.
• Design and maintain real-time streaming pipelines for vehicle telemetry, heartbeats, OTA campaign status, and ECU state changes.
• Build high-throughput data services and workers using Rust for payload routing, processing, and verification.
• Use Apache Kafka and AWS IoT Core for real-time data ingestion and event streaming.
• Leverage AWS MemoryDB for Redis for low-latency fleet state, campaign progression, and session management.
• Build resilient pipelines capable of handling intermittent connectivity, noisy networks, and out-of-order data.
• Build real-time dashboards for fleet health, firmware versions, OTA campaigns, and update progress.
• Define and monitor key OTA metrics including success/failure rates, retry rates, failure categories, and completion time.
• Implement automated alerting and anomaly detection for unexpected failure spikes during staged or canary rollouts.
• Analyze logs, traces, and telemetry data to identify campaign bottlenecks, telemetry loss, and hardware-related failures.

Skills

Rust

AWS

Apache Kafka

MemoryDB

Python

SQL

MQTT

WebSockets

HTTP/S

Distributed systems

Tools

Power BI

Apache Superset

Job description

Experience: 4–6 Years

Domain: Automotive IoT | Connected Vehicles | Firmware & OTA

Core Stack: Rust | AWS | IoT Core | Apache Kafka | MemoryDB

Key Responsibilities

OTA & Firmware Lifecycle

• Co-own OTA firmware rollout operations across multi-ECU connected vehicle architectures.
• Design automated mechanisms to detect update failures, network interruptions, verification errors, and stalled deployments.
• Implement deterministic retry, recovery, and rollback mechanisms to ensure reliable firmware updates without vehicle bricking.
• Ensure firmware package integrity, signature validation, and data security throughout the OTA pipeline.

Data Engineering & Telemetry

• Design and maintain real-time streaming pipelines for vehicle telemetry, heartbeats, OTA campaign status, and ECU state changes.
• Build high-throughput data services and workers using Rust for payload routing, processing, and verification.
• Use Apache Kafka and AWS IoT Core for real-time data ingestion and event streaming.
• Leverage AWS MemoryDB for Redis for low-latency fleet state, campaign progression, and session management.
• Build resilient pipelines capable of handling intermittent connectivity, noisy networks, and out-of-order data.

Monitoring & Analytics

• Build real-time dashboards for fleet health, firmware versions, OTA campaigns, and update progress.
• Define and monitor key OTA metrics including success/failure rates, retry rates, failure categories, and completion time.
• Implement automated alerting and anomaly detection for unexpected failure spikes during staged or canary rollouts.
• Analyze logs, traces, and telemetry data to identify campaign bottlenecks, telemetry loss, and hardware-related failures.

Required Skills

Mandatory

• 4–6 years of experience in Data Engineering, Software Engineering, or IoT Backend Engineering.
• Strong hands‑on experience with Rust for backend/data processing applications.
• Experience with AWS, particularly IoT Core, S3, ECS/EKS, and Lambda.
• Strong experience with Apache Kafka and real‑time data pipelines.
• Hands‑on experience with AWS MemoryDB for Redis or Redis Enterprise.
• Working knowledge of Python and SQL.
• Experience with MQTT, WebSockets, and HTTP/S protocols.
• Strong understanding of distributed systems, streaming data, and resilient data pipelines.

Preferred

• Experience with firmware lifecycle management and OTA systems.
• Exposure to connected vehicles, automotive IoT, telemetry platforms, or connected hardware fleets.
• Experience with device shadows and fleet/device state management.
• Experience building telemetry dashboards using Power BI, Apache Superset, or custom dashboards.
• Experience with staged/canary deployments and automated failure recovery.

Primary Technology Stack

• Languages: Rust, Python, SQL
• Cloud: AWS, IoT Core, S3, ECS/EKS, Lambda
• Streaming: Apache Kafka
• Caching & State: AWS MemoryDB for Redis, Redis
• IoT Protocols: MQTT, WebSockets, HTTP/S
• Analytics & Visualization: Power BI, Apache Superset
• Domain: Automotive IoT, Vehicle Telemetry, OTA, Firmware Management

Skills:- Python, SQL, Internet of Things (IOT), Redis, Message Queuing Telemetry Transport (MQTT), Observability, Log analysis, Data engineering, OTA, Apache Kafka, Apache Superset, PowerBI, Distributed Systems, Systems Development Life Cycle (SDLC), Firmware development, Firmware, memoryDB, Dashboard, connected devices, Microservices, WebSocket, Automotive, Amazon Web Services (AWS) and Rust