Senior Data Engineer

Coforge; Bengaluru; Hybrid; INR 2,500,000 - 4,200,000; Full time

Job summary

Coforge is seeking a Senior Data Engineer/Lead Data Engineer in Bangalore with 7-10 years of experience. The role focuses on Python, PySpark, ETL, SQL, data modeling, and Azure data services. Hybrid work mode, immediate joiner required, and F2F interview in Bangalore.

Candidate should have hands-on experience building scalable data pipelines, data transformations, and cloud-centric data platforms using Azure. OpenShift and CI/CD with GitHub Actions are part of the responsibilities.

Qualifications

• Strong proficiency in Python and PySpark for large-scale data processing and ETL development.
• Experience with SQL for complex querying and data modeling is preferred.
• Hands-on experience with Azure data services and cloud platforms.
• Proficiency in DevOps and CI/CD practices using GitHub Actions.

Responsibilities

• Design, build, and optimize scalable ETL/ELT data pipelines using Python and PySpark in Azure Databricks and Azure Data Factory.
• Develop and manage data ingestion from various sources and transform data for downstream consumption.
• Implement data quality, monitoring, and dashboards (e.g., Grafana) for pipeline health and performance.
• Contribute to OpenShift/Azure infrastructure setup; leverage HELM for deployment.
• Drive CI/CD with GitHub Actions for automated testing, builds, and deployments.
• Develop and optimize SQL queries and data models for efficient storage and retrieval.

Skills

Python

PySpark

ETL

SQL

Data Modeling

Data Warehousing

Azure Databricks

Azure Data Factory

Azure Key Vault

DevOps

CI/CD

OpenShift

Tools

Azure Databricks

Azure Data Factory

Azure SQL Server

Grafana

GitHub Actions

Helm

OpenShift

Kubernetes

Job description

Greetings from Coforge!

Coforge is looking for Senior Data Engineer / Lead Data Engineer with 7-10 years experience in Python, PySpark, ETL, SQL, Data Modeling, Data Warehousing, Azure Databricks, Azure Data Factory, Azure Key Vault, DevOps and CI/CD

Important Note 1:

We are looking for an immediate joiner for this role! If you're on a 30/60/90-day notice period, this opportunity may not be the right fit at the moment. We truly appreciate your understanding!

Important Note 2:

Please apply only if you are available to attend a F2F interview in Bangalore office. Candidates from other locations or those who are unable to attend the F2F interview, kindly excuse.

Please find the below detailed job description and the company profile for your better understanding.

Position: Senior Data Engineer

Experience: 7-10 years

Job Location: Bangalore

Position Type: Full time

Qualification: Any graduate

Work Mode: Hybrid

Interview Mode : F2F(2nd Level)

Job Summary:

We are looking for an experienced Senior Data Engineer with strong expertise in Python, PySpark, ETL, SQL, Data Modeling, Data Warehousing, and Azure Data Engineering services. The ideal candidate should have hands-on experience in building scalable data pipelines, data transformation solutions, and cloud-based data platforms using Azure.

Key Responsibilities:

• 1. Data Pipeline Development: Design, build, and optimize robust, scalable, and efficient ETL/ELT data pipelines using Python and PySpark, primarily within Azure Databricks and Azure Data Factory.
• 2. Data Ingestion & Processing: Develop and manage processes for ingesting data from various sources (e.g., transactional databases, APIs, streaming sources) and transforming it into clean, usable formats for downstream consumption.
• 3. Data Quality & Monitoring: Implement comprehensive unit and integration test coverage for data pipelines. Establish and maintain monitoring, alerting, and dashboarding solutions (e.g., Grafana) for data quality, pipeline health, and performance.
• 4. Cloud Infrastructure Management (OpenShift/Azure): Contribute to the setup, configuration, and maintenance of data-related infrastructure on OpenShift, ensuring deployment readiness and leveraging tools like HELM for application packaging and deployment.
• 5. CI/CD & Automation: Drive CI/CD best practices using GitHub Actions, ensuring automated testing (unit tests), build, and deployment processes for data solutions to environments like OpenShift.
• 6. SQL & Data Modeling: Develop and optimize complex SQL queries for data extraction, transformation, and loading. Apply strong data modeling principles for efficient data storage and retrieval in SQL Server and other data stores.
• 7. Azure Ecosystem Leverage: Utilize a broad range of Azure data and analytics services, including Azure Data Factory, Azure Databricks, Azure SQL Server, Azure Key Vault, and others to build comprehensive data solutions.
• 8. Performance Optimization: Proactively identify and resolve performance bottlenecks in data pipelines and databases through query optimization, indexing strategies, and efficient data processing techniques.
• 9. Collaboration & Documentation: Work closely with data scientists, analysts, and other engineering teams to understand data requirements.
• 10. Create clear and concise documentation for data pipelines, architecture, and processes.

Required Core Skills & Qualifications:

• 1. Programming & Data Processing: Strong proficiency in Python and PySpark for large-scale data processing and ETL development.
• 2. Data Warehousing & SQL: Expertise in SQL for complex querying, data manipulation, and schema design.(Optional, but highly preferred): Proven experience in SQL optimization and performance tuning.
• 3. ETL Development: Demonstrable experience in designing, building, and maintaining robust ETL/ELT data pipelines
• 4. Cloud Data Platform (Azure Focus):Hands-on experience with Azure Databrick.
• 5. Proficiency with core Azure Analytics Services including Azure Data Factory, Azure SQL Server, and Azure Key Vault.
• 6. DevOps & CI/CD: Experience implementing CI/CD pipelines from GitHub (including GitHub Actions) for automated testing (unit tests), build, and deployment processes.
• 7. Containerization & Orchestration: Familiarity and practical experience with OpenShift (setup, deployment-ready configurations, and management).Experience with HELM for deploying applications on Kubernetes/OpenShift.
• 8. Monitoring & Observability: Experience in setting up and configuring Grafana for dashboards to monitor data quality and pipeline health.

Why Coforge?

Coforge is a global digital services and solutions provider that leverages emerging technologies and deep domain expertise to deliver real-world business impact for our clients. Weve emerged as the 7th largest Indian IT services firm.

Who We Are:

Coforge is devoted to making the world a better place for clients, for our communities and for our people.

What We Do:

We help businesses transform into intelligent, high-growth enterprises through capabilities in Generative AI, Cloud, Data & Analytics, Low-Code Platforms, Intelligent Automation and Cybersecurity.

Coforge specializes in Data Governance, Digital Transformation, and Disruptive Technologies, helping clients to capitalize on their potential efficiencies.

Coforge has been an instrumental partner in the digital transformation journey of clients across a broad spectrum of industries: Health Tech, Fin Tech, Hi-Tech, Security, Digital Payments, Education Publication, Travel, Real Estate, Supply Chain and Logistics and Emerging Technologies.

Coforge operate across 25 countries, with 30 global delivery centers, and a thriving global workforce of 40,000+ employees. Our international network ensures that clients receive seamless access to the complete range of our services and expert knowledge and skills of professionals globally.

Coforge has been recognized as a Great Place to Work certified organisation.

Coforge was recognized at the ET Edge Best Organisations for Women 2026.

Interview Levels: 3-Technical Interviews

Regards,

Ravisankar P

Talent Acquisition - Coforge

[phone]

[e-mail]