Data Engineer

Innodata Group; Sector 10; On-site; INR 1,400,000 - 2,200,000; Full time

Job summary

Innodata Group in India is seeking an experienced Data Engineer to design and optimize end-to-end data pipelines using Python and Spark/Databricks, handling structured and semi-structured data at scale.

You will automate workflows with Airflow, manage cloud-native services (AWS Glue, Lake Formation or Azure Data Factory), implement real-time ingestion with Kafka or Kinesis, and maintain relational databases while ensuring data integrity and performance.

Qualifications

• 5-7 years of professional experience in Data Engineering.
• Strong proficiency in Python and SQL.
• Hands-on experience with Spark/Databricks for large-scale processing.
• Experience with AWS, Azure or GCP cloud services and data pipelines.
• Ability to translate technical pipeline architecture into clear business value for stakeholders.

Responsibilities

• Build and optimize end-to-end ETL/ELT pipelines for structured and semi-structured data using Python, PySpark and Databricks.
• Automate complex workflows using Apache Airflow and manage cloud-native services (AWS Glue/Lake Formation or Azure Data Factory).
• Implement real-time ingestion layers using Kafka, AWS MSK, or Kinesis to ensure low-latency data availability.
• Maintain and query relational databases (SQL Server, Oracle) while ensuring high data integrity and performance.
• Manage code versions via Git and contribute to CI/CD pipelines for automated cloud resource deployment.

Skills

Python

SQL

Spark/Databricks

Airflow

Kafka

Git

CI/CD

Cloud (AWS/Azure/GCP)

Tools

Databricks

Apache Airflow

Kafka

AWS Glue

AWS Kinesis

Azure Data Factory

GCP BigQuery

Git

CI/CD pipelines

SQL Server

Oracle

Elasticsearch

Job description

5 - 7 Years

Job Description

Key Responsibilities

• Build and optimize end-to-end ETL/ELT pipelines for structured and semi-structured data using Python, PySpark and Databricks.
• Automate complex workflows using Apache Airflow and manage cloud-native services (AWS Glue/Lake Formation or Azure Data Factory).
• Implement real-time ingestion layers using Kafka, AWS MSK, or Kinesis to ensure low-latency data availability.
• Maintain and query relational databases (SQL Server, Oracle) while ensuring high data integrity and performance.
• Manage code versions via Git and contribute to CI/CD pipelines for automated cloud resource deployment.

Technical Requirements

• 4+ years of professional experience in a Data Engineering role.
• Strong proficiency in Python and SQL.
• Hands-on experience with Spark/Databricks for large-scale processing.
• Proven experience in either AWS (Glue, Kinesis, Lake Formation) or Azure (ADF, Event Hubs), GCP (BigQuery).
• Ability to translate technical pipeline architecture into clear business value for stakeholders.

Good to Have

• Experience with Geospatial data formats and algorithms.
• Experience with AI/ML.
• Experience with Trino SQL.
• Familiarity with NoSQL environments (ElasticSearch, Graph Databases).
• Web fundamentals: Basic knowledge of React/TypeScript for data-heavy applications.

Job Snapshot

Updated Date

03/16/2026

Job ID

Job_147