Data Engineer

InfoVision Inc.; Pune District; On-site; INR 4,000,000 - 6,000,000; Full time

Job summary

InfoVision Inc. in Pune is seeking a seasoned Data Engineer to design, build, and manage enterprise data pipelines on Azure and Databricks.

You will own schema design, API development, and data infrastructure for analytics and intelligence products, collaborating with cross-functional teams to deliver scalable solutions. The role requires 4+ years in data engineering, proficiency with relational and non-relational databases, MongoDB and Cosmos DB, and API development using FastAPI.

Qualifications

• 4+ years data engineering or data platform experience.
• Experience with relational and non-relational databases.
• Expertise in MongoDB data design and aggregation.
• Familiarity with Azure Cosmos DB multi-model APIs and throughput.
• Experience building RESTful APIs and API gateways.

Responsibilities

• Design, build, and manage enterprise data pipelines on Azure and Databricks.
• Own schema design, API development, and data infrastructure for analytics products.
• Monitor pipeline health and implement automated alerting.
• Design relational and non-relational database schemas.
• Collaborate across engineering teams and communicate progress to stakeholders.
• Support integration of prototypes into core products.

Skills

Data engineering

Data modeling

Stakeholder communication

Team collaboration

Azure fundamentals

Tools

Azure Databricks

MongoDB

Azure Cosmos DB

Graph databases

FastAPI

APIM

REST APIs

Job description

Location: Pune | Experience: 4+ years | Type: Full-Time

Role Overview

Design, build, manage enterprise data pipelines on Azure and Databricks. Own schema design, API development, and data infrastructure for analytics and intelligence products.

Key Responsibilities

• Implement robust, scalable data pipelines using Microsoft Azure and Databricks stack
• Build reusable data pipeline components and frameworks
• Design and optimize data workflows for performance and reliability
• Monitor pipeline health and implement automated alerting
• Design relational and non-relational database schemas
• Azure Cosmos DB: multi-model access, partitioning, consistency levels, throughput management (RU/s), global distribution

API Development

• Build RESTful APIs using FastAPI with async endpoint design
• Implement Pydantic models, middleware, dependency injection, background tasks
• Generate and maintain auto-generated OpenAPI/Swagger documentation
• Configure Azure API Management (APIM) for API gateway and security

Project & Stakeholder Management

• Collect progress updates from squads regularly
• Consolidate updates into weekly project status reports
• Develop L3-level project plans with task details, milestones, dependencies
• Participate in early-stage design and feature definition
• Communicate complex data insights to non-technical stakeholders
• Work across multiple engineering teams on prototype integration
• Support integration of proven prototypes into core intelligence products
• Strong team collaboration and cross-functional communication
• Knowledge sharing and documentation

Required Experience:

• 4+ years data engineering or data platform experience
• Relational and non-relational database expertise
• MongoDB: advanced schema design, aggregation, indexing, sharding, performance optimization
• Azure Cosmos DB: multi-model APIs, partitioning, consistency, throughput management
• Graph databases: Cosmos DB Gremlin API, graph modeling, traversals, relationship analytics
• FastAPI proficiency: async endpoints, Pydantic models, middleware, dependency injection
• Background tasks and job scheduling
• RESTful API design and best practices
• Azure fundamentals and hands-on experience
• Azure Data Factory or Databricks for ETL/ELT
• ETL/ELT pipeline design and development
• Data quality validation and monitoring
• Schema design for analytics and reporting
• Performance optimization and scalability