Enable job alerts via email!

Data Engineer - Pyspark

Barclays UK

Chennai

On-site

INR 10,00,000 - 15,00,000

Full time

Yesterday
Be an early applicant

Job summary

A global financial institution is seeking a Data Engineer - PySpark in Chennai. You will build and maintain data architecture, design data warehouses, and work with machine learning models. The ideal candidate has expertise in PySpark and AWS, along with strong SQL and analytical skills. This position emphasizes collaboration and continuous improvement in a dynamic environment.

Qualifications

  • Experience with PySpark and Python.
  • Good understanding of writing and debugging code.
  • Previous experience in banking or financial services domain.

Responsibilities

  • Build and maintain data architecture pipelines.
  • Design and implement data warehouses and data lakes.
  • Collaborate with data scientists to build machine learning models.

Skills

Expertise in PySpark
Strong SQL knowledge
Analytical and problem-solving skills
Excellent communication skills

Tools

AWS (S3, Glue, Athena, etc.)
GIT
Databricks
Snowflake

Job description

Join us as a Data Engineer - PySpark at Barclays, where you will spearhead the evolution of our infrastructure and deployment pipelines, driving innovation and operational excellence. You will harness cutting-edge technology to build and manage robust, scalable, and secure infrastructure, ensuring seamless delivery of our digital solutions.

To be successful as a Data Engineer - PySpark, you should have experience with:

  • Expertise in PySpark, Python.
  • Strong SQL knowledge.
  • Good understanding of writing and debugging code.
  • Quick learner, with strong analytical and problem-solving skills, and excellent written and verbal communication skills.

Some other highly valued skills may include:

  • Good knowledge of AWS cloud services such as S3, Glue, Athena, Lake Formation, CloudFormation, etc.
  • Good understanding of SCM tools like GIT.
  • Previous experience within banking or financial services domain.
  • Experience with Databricks, Snowflake, Starburst, Iceberg.

You may be assessed on key skills relevant for success in this role, such as risk and controls, change and transformation, business acumen, strategic thinking, digital and technology, as well as job-specific technical skills.

The role is based out of Chennai.

Purpose of the role

To build and maintain systems that collect, store, process, and analyze data, such as data pipelines, data warehouses, and data lakes, ensuring data is accurate, accessible, and secure.

Accountabilities

  • Build and maintain data architecture pipelines for data transfer and processing.
  • Design and implement data warehouses and data lakes, managing data volumes and security measures.
  • Develop processing and analysis algorithms suitable for data complexity and volume.
  • Collaborate with data scientists to build and deploy machine learning models.

Analyst Expectations

  • Perform activities timely and to a high standard, driving continuous improvement.
  • Possess in-depth technical knowledge and experience in their area of expertise.
  • Lead and supervise teams, guiding professional development and resource allocation.
  • If in a leadership role, demonstrate leadership behaviors: Listen, Energize, Align, Develop.
  • Develop technical expertise, acting as an advisor.
  • Impact related teams and partner with other functions.
  • Manage operational processing, escalate policy breaches, and embed new policies.
  • Advise on decision-making, manage risks, and ensure compliance with rules and regulations.
  • Understand how their sub-function fits within the organization, making evaluative judgments and resolving problems.
  • Guide team members, communicate effectively, and build stakeholder networks.

All colleagues are expected to demonstrate Barclays Values: Respect, Integrity, Service, Excellence, and Stewardship, and embody the Barclays Mindset: Empower, Challenge, Drive.

Get your free, confidential resume review.
or drag and drop a PDF, DOC, DOCX, ODT, or PAGES file up to 5MB.