Data Engineer

Idexcel; Bengaluru; On-site; INR 900,000 - 1,400,000; Full time

Job summary

Idexcel, Bengaluru-based, seeks a Data Engineer with 4–6 years of hands-on PySpark, Databricks, and AWS experience to design and maintain scalable data pipelines. The role focuses on delivering robust data solutions and data lake integration with strong emphasis on data quality and reliability.

You will collaborate within agile squads, drive technical excellence, and help industrialize data platforms for energy and business operations.

Qualifications

• Master’s degree or engineering school background in CS, Mathematics, or related field.
• 4–6+ years of data engineering experience.
• Experience with PySpark, Databricks, and AWS.
• Strong SQL and data warehousing knowledge.
• Knowledge of data pipelines and ETL/ELT design.

Responsibilities

• Design, build, and maintain robust data pipelines (collection, preparation, transformation, storage, and dissemination of data).
• Ensure data quality, reliability, and scalability for the squad’s product.
• Architect and industrialize data solutions: set up/maintain infrastructure, cloud environments, data flows, and integration into the central data lake.
• Analyze data sources, assess accessibility, and recommend integration approaches.
• Implement monitoring, automated testing, and best practices for data flow reliability.
• Produce clean, high-quality, testable code.
• Provide technical recommendations and interact with TL and other Data Engineers.
• Coach and support the broader Data Engineering community and contribute to best-practice standards.

Skills

Python

PySpark

Databricks

Unity Catalog

AWS Cloud

SQL

Data Warehousing

ETL/ELT

Databases (RDBMS/NoSQL)

DDD / Software Craft

Education

Master's degree or engineering degree in Computer Science/Mathematics

Tools

Unity Catalog

CI/CD pipelines

Infrastructure as Code

Git

Performance tuning

Cost optimization

Job description

Role & responsibilities

Position Title Data Engineer(PySpark / Databricks / Unity Catalog, AWS)

Overall exp – 4 to 6+ years

Qualification Criteria- Master’s degree or engineering school background in Computer Science, Mathematics, or a related field

Mandatory Skills

• Python Development
• PySpark & Databricks
• AWS Cloud
• SQL & Data Warehousing
• Data Pipeline & ETL/ELT Engineering

ONLY IMMEDIATE JOINERS REQUIRED

Work Location – Bangalore

Job Description:

Responsibilities

Core Technical Duties

Design, build, integrate, and maintain robust data pipelines (collection, preparation, transformation, storage, and dissemination of data). Ensure data quality, reliability, and scalability for the squad’s product. Architect and industrialize data solutions: set up/maintain infrastructure, cloud environments, data flows, and integration into the central data lake. Analyze data sources, assess accessibility, and recommend integration approaches. Implement monitoring, automated testing, and best practices for data flow reliability. Produce clean, high-quality, testable code.

Transversal / Leadership Duties (Senior Level)

Provide technical recommendations and interact with TL and other Data Engineers. Conduct technological watch (staying current on data architectures, tools, and emerging IA technologies). Coach and support the broader Data Engineering community (e.g., via Communities of Practice/CoP). Contribute to knowledge sharing and best-practice standards across squads.

The focus is on industrializing data solutions — moving from prototypes to reliable, production-grade systems that support the energy transition and business operations.

Day-to-Day Elements include:

• Data Engineering & Pipeline Development
• Data Products Enablement
• Platform & Architecture Implementation
• Data Quality, Reliability & Operations
• Collaboration & Agile Delivery

Required Skills

Technical Skills (Core Stack)

• Strong mastery of: Python, PySpark / Databricks / Unity Catalog, AWS, SQL.
• Solid knowledge of relational and non-relational databases.
• Experience designing/implementing large-scale volumetry data loading, real-time, processing, analysis, and exploration solutions.
• Experience with cloud platforms (AWS preferred; Azure also relevant).
• Best practices in data pipeline monitoring, testing, and quality assurance.
• Software Craft & DDD knowledge
• GEN IA daily use (skills, MCP, etc.)

Soft/Additional Skills

• Ability to work in agile, cross-functional teams.
• Strong collaboration and communication skills.
• Interest in coaching others and driving technical excellence.

Tools/software

CI/CD pipelines, InfrastructureasCode concepts , Version control (Git), Experience with performance tuning and cost optimization