Lead Software Engineer - Data Ingestion

Societe Generale SA; Bengaluru; Hybrid; INR 900,000 - 1,300,000; Full time

Job summary

Societe Generale in Bengaluru is seeking a Medior Lead Software Engineer for Data Ingestion to develop and operate data ingestion platforms orchestrated by Apache Airflow. You will onboard and maintain data integrations from various sources, troubleshoot issues, and support governance processes.

The role focuses on building ETL/ELT pipelines, tuning performance, and collaborating with architects and platform owners in a hybrid work setting in Bangalore.

Qualifications

• 2+ years of Python development experience.
• Hands-on experience with Apache NiFi and Apache Airflow.
• Experience integrating data from relational databases and streaming systems.
• Understanding of ETL/ELT concepts and performance tuning.
• Knowledge of Linux environments and basic scripting.

Responsibilities

• Configure and maintain data ingestion pipelines.
• Develop Python-based ingestion components and utilities.
• Integrate data from relational databases, Kafka, and cloud storage.
• Troubleshoot data flow, connectivity, and integration issues.
• Monitor pipelines, assist with deployments, and governance.
• Create and maintain technical documentation and runbooks.

Skills

Python

NiFi

Airflow

Relational DBs

ETL/ELT

Kafka

Elasticsearch

SQL

Incident Mgmt

Documentation

Tools

Jira

Git

CI/CD

Linux

Job description

# Lead Software Engineer - Data IngestionApply---Bangalore, IndiaHybridReference 26000KFAStart date ImmediatelyPublication date 2026/10/05## Responsibilities**Data Ingestion Specialist****Role Summary**We are looking for a Medior Data Ingestion Specialist to support development, operation, and maintenance of data ingestion platform orchestrated by Apache Airflow. The role focuses on onboarding and maintaining data integrations from multiple source systems, troubleshooting operational issues, and supporting release and governance processes**Key Responsibilities****Data Integration & Pipeline Development*** Configure and maintain data ingestion pipelines* Develop and support Python-based ingestion components* Integrate data: + Relational databases (Oracle, PostgreSQL, MS SQL, MySQL, DB2, etc) + Apache Kafka + AWS S3, flat files + Elasticsearch, Hive, Impala, hBase* Troubleshoot data flow, connectivity, and integration issues* Ingestion to Databricks, Databricks loader**Platform Operations*** Support incident, problem, and change management activities* Maintain Python libraries and dependencies* Create ETL/ELT pipelines focused on source ingestion* Monitor pipeline execution and resolve operational issues* Assist with platform upgrades, patching, and maintenance activities**Release & Governance*** Support Dev/Test/Prod deployment processes* Review configuration and template changes* Ensure compliance with governance, security, and operational standards* Maintain technical and operational documentation**Collaboration*** Work closely with architects, engineering teams, and platform owners* Collaborate with security, audit, and infrastructure teams* Participate in release planning and communication activities**Nice-to-Have Skills*** Knowledge of ETL Python frameworks (e.g. Meltano)* Experience with other ETL/ELT tools* Experience with Git, CI/CD pipelines, and DevOps practices* Knowledge of Linux environments* Experience with Jira and ITSM tooling* Exposure to cloud-based data platforms**Candidate Profile*** Medior-level engineer capable of working independently on standard tasks* Able to create and resolve customers' requirements* Able to analyse integration issues and identify root causes* Comfortable working in an enterprise environment with governance and security requirements* Good communication skills and ability to collaborate with multiple stakeholders## Profile required**Must-Have Skills*** 2+ years of experience with Python development* Practical (hands on) experience with Apache NiFi* Practical (hands on) experience with Apache Airflow* Experience integrating data from relational databases* ETL/ELT performance tuning, optimalizations* Basic experience with Apache Kafka* Basic experience with Elasticsearch* SQL knowledge and understanding of data integration concepts* Experience troubleshooting technical and connectivity issues* Understanding of incident and release management processes* Ability to create and maintain technical documentation* Strong analytical and problem-solving skills## Why join us*“We are committed to creating a diverse environment and are proud to be an equal opportunity employer. All qualified applicants receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status”.*## Business insight**At Société Générale, we are convinced that people are drivers of change, and that the world of tomorrow will be shaped by all their initiatives, from the smallest to the most ambitious. Whether you’re joining us for a period of months, years or your entire career, together we can have a positive impact on the future. Creating, daring, innovating, and taking action are part of our DNA. If you too want to be directly involved, grow in a stimulating and caring environment, feel useful on a daily basis and develop or strengthen your expertise, you will feel right at home with us!****Still hesitating?****You should know that our employees can dedicate several days per year to solidarity actions during their working hours, including sponsoring people struggling with their orientation or professional integration, participating in the financial education of young apprentices, and sharing their skills with charities. There are many ways to get involved.****We are committed to support accelerating our Group’s ESG strategy by implementing ESG principles in all our activities and policies. They are translated in our business activity (ESG assessment, reporting, project management or IT activities), our work environment and in our responsible practices for environment protection.**We are an **equal opportunities employer** and we are proud to make diversity a strength for our company. Societe Generale is committed to recognizing and **promoting all talents**, regardless of their beliefs, age, disability, parental status, ethnic origin, nationality, gender identity, sexual orientation, membership of a political, religious, trade union or minority organisation, or any other characteristic that could be subject to discrimination.