Data Engineer Pyspark

Citi

Hyderabad

On-site

INR 1,200,000 - 1,800,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Citi is recruiting for an Applications Development Intermediate Programmer Analyst in Hyderabad, India. The role focuses on hands-on data engineering, building batch and real-time pipelines, and optimizing processing using Spark, Databricks, Kafka, and SQL.

You will work with a data‑engineering team, ensuring secure, scalable solutions while adhering to governance and production support standards. This is a full‑time position escalating with senior colleagues.

Qualifications

  • Hands-on experience in application development, data engineering, or distributed data processing using Apache Spark and Databricks.
  • Proficiency in Scala, Python, and SQL, including developing and optimizing data-processing workflows and queries.
  • Working knowledge of Spark concepts such as execution plans, partitioning, shuffling, joins, caching, and memory utilization.
  • Experience with Kafka, Spark Structured Streaming, Apache Iceberg, Delta Lake, or similar data and streaming technologies.

Responsibilities

  • Develop, test, enhance, and support batch and real-time data pipelines using Spark, Databricks, Kafka, Scala, Python, and SQL.
  • Write clean, reusable code and participate in code reviews, unit testing, integration testing, data validation, and deployment activities.
  • Analyze Spark execution plans, DAGs, metrics, and logs to troubleshoot failures and improve processing performance.
  • Build and maintain Iceberg or Delta Lake solutions using Lakehouse and medallion patterns.

Skills

Apache Spark
Databricks
Kafka
Scala
Python
SQL

Education

Bachelor’s degree/University degree or equivalent experience

Job description

The Applications Development Intermediate Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming activities.

This is a hands‑on development role responsible for building, testing, optimizing, and supporting batch and real‑time data‑processing solutions using technologies such as Apache Spark, Databricks, Kafka, Scala, Python, and SQL. The successful candidate will work within a data‑engineering team to deliver reliable and maintainable data pipelines, troubleshoot performance and production issues, and follow established architecture, coding, testing, security, governance, and operational standards under the guidance of senior developers and technical leads.

Responsibilities
  • Develop, test, enhance, and support batch and real‑time data pipelines using Apache Spark, Databricks, Kafka, Scala, Python, and SQL.
  • Write clean, reusable code and participate in code reviews, unit testing, integration testing, data validation, and deployment activities.
  • Analyze Spark execution plans, DAGs, metrics, and logs to troubleshoot failures and improve processing performance.
  • Build and maintain Iceberg or Delta Lake solutions using established Lakehouse and medallion architecture patterns.
  • Collaborate with technical leads, developers, analysts, and support teams to deliver assigned work and resolve production issues while following established controls and standards.
  • Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements, including using script tools and analyzing/interpreting code
  • Consult with users, clients, and other technology groups on issues, and recommend programming solutions, install, and support customer exposure systems
  • Apply fundamental knowledge of programming languages for design specifications.
  • Analyze applications to identify vulnerabilities and security issues, as well as conduct testing and debugging
  • Serve as advisor or coach to new or lower‑level analysts
  • Identify problems, analyze information, and make evaluative judgements to recommend and implement solutions
  • Resolve issues by identifying and selecting solutions through the applications of acquired technical experience and guided by precedents
  • Has the ability to operate with a limited level of direct supervision.
  • Can exercise independence of judgement and autonomy.
  • Acts as SME to senior stakeholders and /or other team members.
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm’s reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
  • Independently exercise judgement and decision‑making in managing day‑to‑day responsibilities, including evaluating complex situations, resolving issues, and determining appropriate courses of action. Drive and influence delivery of work within area of responsibility, contributing to operational priorities and ensuring progress against timelines. Will be accountable for delivery outcomes, including quality and timelines, and may have a measurable impact on business results within scope. In addition, will provide oversight on execution by reviewing outputs, guiding approach where required, and addressing gaps to ensure delivery meets expected standards. Take ownership of decisions and deliverables within remit, applying analytical thinking and professional expertise to solve problems and ensure effective outcomes.
  • Support execution of business decisions within the function by translating direction into actionable steps and ensuring effective follow‑through. Address non‑routine situations and practical challenges by applying judgement to identify solutions or take decisions within scope. Will support effective functioning of the business area by ensuring processes are implemented consistently and outputs meet expected standards. Operate across varied and evolving situations, exercising discretion to respond to changing requirements and priorities, while ensuring reliability, timeliness, and quality in execution within defined scope.
  • Applies domain expertise and advanced analytical capability to deliver outputs aligned with established standards and professional benchmarks. Evaluate business requirements by analyzing complex issues, forming well‑reasoned views, and recommending practical, implementable solutions. Apply structured thinking, analytical rigor, and creativity in approaching work, particularly in situations where standard methodologies may require refinement. Exercise sound professional judgement in interpreting requirements and determining appropriate courses of action, taking into account the broader business context and potential implications.
Qualifications
  • Hands‑on experience in application development, data engineering, or distributed data processing using Apache Spark and Databricks.
  • Proficiency in Scala, Python, and SQL, including developing and optimizing data‑processing workflows and queries.
  • Working knowledge of Spark concepts such as execution plans, partitioning, shuffling, joins, caching, and memory utilization.
  • Experience with Kafka, Spark Structured Streaming, Apache Iceberg, Delta Lake, or similar data and streaming technologies.
  • Understanding of cloud platforms, Lakehouse and medallion architecture, version control, testing, deployment, monitoring, and production support.
  • 5-8 years of relevant experience in the Financial Service industry
  • Intermediate level experience in Applications Development role
  • Consistently demonstrates clear and concise written and verbal communication
  • Demonstrated problem‑solving and decision‑making skills
  • Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
Education
  • Bachelor’s degree/University degree or equivalent experience

This job description provides a high‑level review of the types of work performed. Other job‑related duties may be assigned as required.

------------------------------------------------------

Job Family Group:

Technology

------------------------------------------------------

Job Family:

Applications Development

------------------------------------------------------

Time Type:

Full time

------------------------------------------------------

Most Relevant Skills

Please see the requirements listed above.

------------------------------------------------------

Other Relevant Skills

For complementary skills, please see above and/or contact the recruiter.

------------------------------------------------------

Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.

If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi. View Citi’s EEO Policy Statement and the Know Your Rights poster.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Big Data pyspark developer
Big Data pyspark developer

Citi • India

On-site
INR 1,000,000 - 1,700,000
Big Data Java Spark Developer
Big Data Java Spark Developer

Citi • Pune District

On-site
INR 2,800,000 - 4,200,000
Big Data and Apache Spark Developer - Assistant Vice President
Big Data and Apache Spark Developer - Assistant Vice President

Mployee.me • Pune District

On-site
INR 1,800,000 - 3,000,000
Big Data pyspark developer
Big Data pyspark developer

Citi • Pune

On-site
INR 900,000 - 1,300,000
Big Data Engineering AVP
Big Data Engineering AVP

Citi • Pune District

On-site
INR 1,800,000 - 3,000,000
Senior Data Engineering- Java Spark
Senior Data Engineering- Java Spark

Mployee.me • Pune District

On-site
INR 2,800,000 - 4,000,000
Big Data and Apache Spark Developer - Assistant Vice President
Big Data and Apache Spark Developer - Assistant Vice President

Citi • Pune District

On-site
INR 3,000,000 - 6,000,000
Data Engineer, Java and Oracle - Assistant Vice President
Data Engineer, Java and Oracle - Assistant Vice President

Citi • Pune District

On-site
INR 1,200,000 - 1,800,000
Java spark Developer
Java spark Developer

Citigroup Inc. • Chennai

On-site
INR 1,500,000 - 2,800,000
Pyspark Engineer
Pyspark Engineer

Citi • India

On-site
INR 900,000 - 1,500,000