Java Scala Developer_ Apache Spark & SQL

Citigroup; Pune District; In loco; INR 4.000.000 - 7.000.000; Tempo pieno

Descrizione del lavoro

Citigroup seeks a Senior Big Data Engineer to design, develop, and optimize large-scale data processing pipelines using Apache Spark, Spark SQL, Java/Scala, Spring Boot, and Python/PySpark. The role emphasizes distributed data processing, complex query optimization, and data analytics across regulatory datasets.

You will work with modern CI/CD workflows and GenAI tooling to accelerate prototyping, improve code quality, and translate business requirements into robust high- and low-level designs.

Competenze

• 5-9 yrs experience in applications development and data engineering roles.
• Strong communication skills, verbal and written.
• Proficient in Java/Scala, Spark, Spark SQL, and Python/PySpark.

Mansioni

• Architect and implement scalable data pipelines using Spark, Java/Scala, and PySpark.
• Design high-level and low-level designs (HLD/LLD) from business requirements.
• Develop and optimize Spark SQL queries for large datasets.
• Collaborate with stakeholders and deliver CI/CD driven deployments.
• Apply data analysis to derive insights and ensure data quality.
• Leverage GenAI tools to accelerate delivery and improve code quality.

Conoscenze

Java

Scala/Python

Apache Spark

Spark SQL

Python/PySpark

Tableau

AI tool usage

Data analysis

Formazione

Bachelor's degree

Strumenti

Git

Jenkins

Lightspeed

Harness

Spring Boot

GitHub Copilot

Devin AI

Descrizione del lavoro

The Applications Development Intermediate Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming activities.

Responsibilities:

• Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements, including using script tools and analyzing/interpreting code
• Consult with users, clients, and other technology groups on issues, and recommend programming solutions, install, and support customer exposure systems
• Apply fundamental knowledge of programming languages for design specifications.
• Analyze applications to identify vulnerabilities and security issues, as well as conduct testing and debugging
• Serve as advisor or coach to new or lower level analysts
• Identify problems, analyze information, and make evaluative judgements to recommend and implement solutions
• Resolve issues by identifying and selecting solutions through the applications of acquired technical experience and guided by precedents
• Has the ability to operate with a limited level of direct supervision.
• Can exercise independence of judgement and autonomy.
• Acts as SME to senior stakeholders and /or other team members.
• Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.

Qualifications:

5- 9 yrs Experienced resource with

• Strong communication skills - both verbal and written
• Strong in Java/Scala development, spring , spring boot, Apache Spark, Spark SQL, shell scripting
• Ability to work in Python is also required. Pyspark experience preferable.
• Strong in SQL programming especially spark SQL
• Demonstrated Data analysis skills namely - data sourcing , processing and insights.
• Tableau experience is good to have but not mandatory
• Expected experience should be minumum 7+ years
• Ability to understand requirements and develop high level and low level design
• Has experience in devops tools like github, Jenkins/lightspeed, harness
• Has experience in using AI tools like github copilot, DevinAI, Cluade code

Experience Level

5 to 9 years as discussed

Key Skills

• Java
• Scala/python
• Apache Spark
• Spark SQL
• AI tool usage based development - DevinAI, github copilot etc.

Good to have skills

• Python
• Unix shell scripting
• Tableau for visualization and reporting

Education

• Bachelor's degree/University degree or equivalent experience

This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.

We are seeking an experienced and forward-thinking Senior Big Data Engineer (7+ Years) to join our technology team. In this role, you will be responsible for designing, developing, and optimizing high-performance, large-scale data processing pipelines and regulatory data platforms using Apache Spark, Java/Scala, Spring Boot, and Python/PySpark.

The ideal candidate brings deep expertise in distributed data computing, complex Spark SQL query optimization, data analysis, and modern CI/CD DevOps workflows. You will actively leverage GenAI engineering tools (such as GitHub Copilot and Devin AI) to accelerate delivery, drive code quality, and translate complex business requirements into robust High-Level (HLD) and Low-Level Designs (LLD).

Key Responsibilities

• Architect & Develop Data Solutions: Design, implement, and maintain scalable, high-throughput batch and stream data pipelines using Apache Spark, Java/Scala, and PySpark.
• End-to-End System Design: Analyze business and regulatory requirements to create comprehensive High-Level Designs (HLD), Low-Level Designs (LLD), data flow diagrams, and interface specifications.
• Data Engineering & Analytics: Perform in-depth data analysis across the data lifecycle-including data sourcing, schema evolution, transformation logic, quality validation, and actionable insight generation.
• Spark SQL Optimization: Write, tune, and maintain complex Spark SQL queries, partitioned datasets, and transformations for optimal performance over large enterprise datasets.
• Microservices & API Integration: Build enterprise-grade services and data ingestion microservices using Spring and Spring Boot.
• DevOps & CI/CD Delivery: Implement automated build, test, and deployment workflows using Git/GitHub, Jenkins / Lightspeed, and Harness.
• AI-Augmented Engineering: Leverage modern GenAI developer tooling (e.g., GitHub Copilot, Devin AI) to accelerate prototyping