Data Engineering Lead

慨正橡扯; Pune District; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

Barclays is seeking a Data Engineering Lead based in Pune to spearhead cutting-edge data engineering initiatives. You will manage cloud data platforms, ensuring accurate and secure data while collaborating with diverse teams to shape data strategy. The ideal candidate will have expertise in Python, SQL, and modern data frameworks.

This role emphasizes leadership in driving consistent engineering practices, mentoring teams, and ensuring compliance within a regulated environment, supporting both technical and strategic business objectives.

Qualifications

• Experience in leading data engineering projects in regulated environments.
• Strong understanding of data quality, metadata, and lineage.
• Ability to influence and coach engineers towards excellence.

Responsibilities

• Build and maintain data architectures and pipelines.
• Design data warehouses and lakes adhering to security measures.
• Collaborate with data scientists for machine learning model deployment.

Skills

Large scale data engineering initiatives on cloud data platforms

End-to-end data delivery accountability

Standardized engineering practices

Partnering with stakeholders for data strategy

Exposure in Abinitio and Oracle AWS technologies

DBT, Databricks, Airflow

Deep expertise in Python and SQL

Education

Relevant technical degree

Tools

Abinitio

Oracle

AWS

DBT

Databricks

Airflow

Job description

Join us as a Data Engineering Lead. At Barclays, innovation isn’t just encouraged; it’s expected. You will be required to have an experience in SOLD Simplification. It is a multiyear initiative to make the platform compliant to most valuable environment, hardware refresh and removing redundant functionalities. You will be required to lead the overall engineering development of this initiative. You will be an Engineering Lead and required to coordinate support requirement Discovery, initiative delivery planning, roadmap and stakeholder management.

To be successful as a Data Engineering Lead you should have experience with:

• Leading large scale data engineering initiatives on modern cloud data platforms, delivering resilient, governed, and audit ready data products in regulated banking environments.
• Owning end to end data delivery across ingestion, transformation, quality, lineage, and consumption, with strong accountability for reliability, SLAs, and production stability.
• Driving adoption of standardized engineering practices (SDLC, CI/CD, controls, documentation) across multiple teams and portfolios within an enterprise data platform.
• Partnering with Product, Risk, Technology Control, and Business stakeholders to shape data strategy and translate regulatory and business outcomes into scalable architectures.
• Exposure in Abinitio, Oracle AWS technologies.
• Understanding of the Cards ecosystem.
• Working knowledge of DBT, Databricks and Airflow.
• Deep expertise in Python, SQL, and modern data engineering frameworks, with a track record of building and governing complex transformation pipelines at scale.

Some other highly valued skills may include:

• Strong understanding of data quality, metadata, lineage, and observability, and the ability to institutionalize these as platform capabilities rather than team specific solutions.
• Experience defining and enforcing engineering standards and architectural guardrails across lake house style or cloud native data platforms.
• Proven leadership in coaching senior engineers, influencing across tribes, and driving engineering excellence in a federated enterprise model.

You may be assessed on the key critical skills relevant for success in role, such as risk and controls, change and transformation, business acumen strategic thinking and digital and technology, as well as job‑specific technical skills.

This role is based in our Pune office.

Purpose of the role

To build and maintain the systems that collect, store, process, and analyse data, such as data pipelines, data warehouses and data lakes to ensure that all data is accurate, accessible, and secure.

Accountabilities

• Build and maintenance of data architectures pipelines that enable the transfer and processing of durable, complete and consistent data.
• Design and implementation of data warehoused and data lakes that manage the appropriate data volumes and velocity and adhere to the required security measures.
• Development of processing and analysis algorithms fit for the intended data complexity and volumes.
• Collaboration with data scientist to build and deploy machine learning models.

Vice President Expectations

• To contribute or set strategy, drive requirements and make recommendations for change. Plan resources, budgets, and policies; manage and maintain policies/ processes; deliver continuous improvements and elevate breaches of policies/procedures.
• If managing a team, they define jobs and responsibilities, planning for the department’s future needs and operations, counselling employees on performance and contributing to employee pay decisions/changes. They may also lead a number of specialists to influence the operations of a department, in alignment with strategic as well as tactical priorities, while balancing short and long term goals and ensuring that budgets and schedules meet corporate requirements.
• If the position has leadership responsibilities, People Leaders are expected to demonstrate a clear set of leadership behaviours to create an environment for colleagues to thrive and deliver to a consistently excellent standard. The four LEAD behaviours are: L Listen and be authentic, E Energise and inspire, A Align across the enterprise, D Develop others.
• OR for an individual contributor, they will be a subject matter expert within own discipline and will guide technical direction. They will lead collaborative, multi‑year assignments and guide team members through structured assignments, identify the need for the inclusion of other areas of specialisation to complete assignments. They will train, guide and coach less experienced specialists and provide information affecting long term profits, organisational risks and strategic decisions.
• Advise key stakeholders, including functional leadership teams and senior management on functional and cross functional areas of impact and alignment.
• Manage and mitigate risks through assessment, in support of the control and governance agenda.
• Demonstrate leadership and accountability for managing risk and strengthening controls in relation to the work your team does.
• Demonstrate comprehensive understanding of the organisation functions to contribute to achieving the goals of the business.
• Collaborate with other areas of work, for business aligned support areas to keep up to speed with business activity and the business strategies.
• Create solutions based on sophisticated analytical thought comparing and selecting complex alternatives. In-depth analysis with interpretative thinking will be required to define problems and develop innovative solutions.
• Adopt and include the outcomes of extensive research in problem solving processes.
• Seek out, build and maintain trusting relationships and partnerships with internal and external stakeholders in order to accomplish key business objectives, using influencing and negotiating skills to achieve outcomes.