Senior Data Engineer (AI Data Platforms & Automation)

E Solutions; Dadri; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

E Solutions seeks an experienced Senior Data Engineer to build scalable data platforms and AI-ready data pipelines across our growing organization. You will architect data flows, implement robust ETL/ELT processes, and drive automation for faster data delivery.

You will design data architectures, metadata frameworks, data models, and governance practices while collaborating with data science and product teams to support AI initiatives at scale.

Qualifications

• Experience designing scalable ETL/ELT pipelines.
• Experience with data warehousing and data modeling.
• Knowledge of metadata management and data governance.
• Experience with cloud platforms (GCP or AWS).
• Airflow experience preferred.
• Strong debugging and automation skills.

Responsibilities

• Design and develop scalable ETL/ELT pipelines.
• Build robust data ingestion frameworks.
• Design enterprise data models.
• Implement data transformation and validation processes.
• Optimize data storage and retrieval.
• Develop automation tooling and dashboards.

Skills

Python

SQL

PySpark

Tools

Docker

Git

CI/CD

API Integration

Job description

We are looking for an experienced and hands-on Senior Data Engineer to build scalable data platforms and AI-ready data pipelines. The ideal candidate will be responsible for designing data architectures, metadata frameworks, ETL pipelines, and automation that support AI and data initiatives across the organization.

Key Responsibilities

Data Engineering

• Design and develop scalable ETL/ELT pipelines.
• Build robust data ingestion frameworks.
• Design enterprise data models.
• Implement data transformation and validation processes.
• Optimize data storage and retrieval.

AI Data Platforms

• Build AI-ready datasets.
• Design metadata frameworks and taxonomies.
• Develop dataset versioning strategies.
• Manage structured and unstructured datasets.
• Support AI model training and evaluation pipelines.

Data Quality & Governance

• Implement data quality checks.
• Build validation and monitoring frameworks.
• Define data standards and best practices.
• Ensure data integrity and consistency.
• Develop documentation and data lineage.

Automation

• Automate repetitive data workflows.
• Build Python-based automation tools.
• Develop integrations with internal and external systems.
• Create dashboards and monitoring utilities.

Platform Engineering

• Work with SQL and NoSQL databases.
• Optimize database performance.
• Manage cloud-based data platforms.
• Support scalable enterprise data architecture.

Required Technical Skills

Programming

• Python (Expert)
• SQL (Expert)
• PySpark (Preferred)

Databases

• PostgreSQL
• MySQL
• MongoDB
• BigQuery / Snowflake
• Vector databases (Preferred)

Data Engineering

• ETL/ELT Development
• Data Warehousing
• Data Modeling
• Metadata Management
• Data Quality Frameworks

Cloud

• Google Cloud Platform or AWS
• Cloud Storage
• BigQuery / Redshift
• Airflow (Preferred)

Tools

• Docker
• Git
• CI/CD
• API Integration

Preferred Experience

• AI data platforms
• Metadata management
• Dataset curation
• Workflow automation
• Enterprise data engineering
• Cloud-native data platforms
• AI data preparation