Data Streaming Specialist

Navikenz India; Bengaluru; Híbrido; INR 1.400.000 - 2.800.000; Tempo integral

Resumo da vaga

Navikenz India is seeking a senior data engineer to design and implement real-time and batch data pipelines, build scalable data platforms, and collaborate with AI/ML teams. You will work across on-premises and hybrid environments, leveraging Kafka, Spark, Flink, Airflow, and dbt to ensure data quality and observability.

The role requires 6+ years in data engineering with strong Java/SQL skills and experience with data lakes/warehouses such as Delta Lake, Snowflake, or BigQuery.

Qualificações

• Bachelors or Masters in Computer Science, Engineering, or related field.

Responsabilidades

• Design and implement real-time (Kafka, Spark/Flink) and batch (Airflow, Spark) pipelines for high-throughput data ingestion, processing, and transformation.

Conhecimentos

Java

SQL

Apache Kafka

Apache Spark

Apache Flink

Airflow

dbt

Terraform

Kubernetes

Data modeling

Formação

Bachelor’s/Master’s in CS/Engineering

Ferramentas

Delta Lake

Iceberg

Snowflake

BigQuery

Redshift

Descrição da vaga

Key Responsibilities

• Design and implement real-time (Kafka, Spark/Flink) and batch (Airflow, Spark) pipelines for high-throughput data ingestion, processing, and transformation.
• Develop data models and manage data lakes and warehouses (Delta Lake, Iceberg, etc) to support both analytical and ML workloads.
• Integrate data from diverse sources: IoT sensors, databases (SQL/NoSQL), REST APIs, and flat files.
• Ensure pipeline scalability, observability, and data quality through monitoring, alerting, validation, and lineage tracking.
• Collaborate with AI/ML teams to provision clean and ML-ready datasets for training and inference.
• Deploy, optimize, and manage pipelines and data infrastructure across on-premise and hybrid environments.
• Participate in architectural decisions to ensure resilient, cost-effective, and secure data flows.
• Contribute to infrastructure-as-code and automation for data deployment using Terraform, Ansible, or similar tools.

Qualifications & Required Skills

• Bachelors or Masters in Computer Science, Engineering, or related field.
• 6+ years in data engineering roles, with at least 2 years handling real-time or streaming pipelines.
• Strong programming skills in Java and SQL.
• Experience with Apache Kafka, Apache Spark, or Apache Flink for real-time and batch processing.
• Hands-on with Airflow, dbt, or other orchestration tools.
• Familiarity with data modeling (OLAP/OLTP), schema evolution, and format handling (Parquet, Avro, ORC).
• Experience with hybrid/on-prem and cloud platforms (AWS/GCP/Azure) deployments.
• Proficient in working with data lakes/warehouses like Snowflake, BigQuery, Redshift, or Delta Lake.
• Knowledge of DevOps practices, Docker/Kubernetes, Terraform or Ansible.
• Exposure to data observability, data cataloging, and quality tools (e.g., Great Expectations, OpenMetadata).