PySpark Big Data Developer

Citibank (Switzerland) AG; Pune District; On-site; Confidential; Full time

Job summary

Citibank (Switzerland) AG in Pune, Maharashtra, India, is seeking a PySpark Big Data Developer to design, develop, and optimize scalable data pipelines for large-scale analytics. The role emphasizes PySpark coding, collaboration with cross-functional teams, and ensuring data integrity throughout the lifecycle.

Ideal candidates will have 4-8 years of enterprise application development, strong PySpark and Big Data experience, and familiarity with SQL Server and Oracle databases.

Qualifications

• 4-8 years of enterprise application development experience.
• Strong Python and PySpark proficiency.
• Hands-on experience with Big Data stack (HDFS, Hive, Sqoop, Hadoop).
• Good exposure to SQL Server and Oracle databases.

Responsibilities

• Design & develop scalable PySpark ETL pipelines for large datasets.
• Write clean, well-documented Python code and leverage PySpark frameworks.
• Collaborate with cross-functional teams to understand data requirements.
• Debug and optimize Spark jobs; troubleshoot data processing bottlenecks.
• Participate in the full SDLC from requirements to deployment and operations.
• Ensure data quality and integrity throughout the data lifecycle.

Skills

PySpark

OOP

Shell Scripting

Git

JIRA

Confluence

Autosys

Tableau

Tools

HDFS

Hive

Sqoop

Hadoop

SQL Server

Oracle

CI/CD

Job description

## PySpark Big Data DeveloperApplyremote type: Hybridlocations: Pune Maharashtra Indiatime type: Full timeposted on: Posted 8 Days Agojob requisition id: 26971489We are seeking a highly skilled and experienced BigData/PySpark Engineer to join our dynamic Big Data Analytics team. This role is pivotal in designing, developing, and optimizing robust, scalable data pipelines for large-scale data processing and analytics.Key Responsibilities:* Design & Development: Create and optimize scalable ETL (Extraction, Transformation, Loading) pipelines using PySpark for massive datasets.* Coding & Engineering: Write clean, efficient, well-documented code primarily in Python (PySpark) often leveraging frameworks/tools.* Collaboration: Work with cross-functional teams (senior developers, data engineers, analysts, business partners) to understand data requirements and ensure seamless solution integration.* Troubleshooting & Optimization: Debug and resolve data processing issues and performance bottlenecks in Spark applications and other big data technologies.* Full SDLC Involvement: Participate in the entire software development lifecycle, from requirements analysis and design to testing, deployment, and operations.* Data Integrity: Ensure high data quality and integrity throughout the data lifecycle.This candidate possesses **4-8 years of experience** in developing and managing Enterprise Applications, demonstrating a robust foundation in Big Data technologies and a strong grasp of software development principles.**Key Experience & Expertise:*** **Enterprise Application Development:** 4-8 years in developing and managing enterprise-grade applications.* **Object-Oriented Programming (OOP):** Solid foundation in OOP concepts.* **Big Data Development:** Expertise in **PySpark, HDFS, Hive, Sqoop, and Hadoop** for Big Data environments.* **Database Technologies:** Good exposure to **SQL Server** and **ORACLE** databases. Experience with query writing for data validation/manipulation* **Scripting & Automation:** Proficient in **Shell Scripting** and experience with job scheduling tools like **Autosys**.* **BI Reporting Tools:** Some exposure to BI tools, specifically **Tableau**.* **Tools & Practices:** Proficient with Git; experience with JIRA, Confluence. Familiarity with DevOps and CI/CD pipelines.------------------------------------------------------## **Job Family Group:**Technology------------------------------------------------------## **Job Family:**Applications Development------------------------------------------------------## **Time Type:**Full time------------------------------------------------------## **Most Relevant Skills**Please see the requirements listed above.------------------------------------------------------## **Other Relevant Skills**For complementary skills, please see above and/or contact the recruiter.------------------------------------------------------*Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.**If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review* *Accessibility at Citi.* *View Citi’s EEO Policy Statement and the Know Your Rights poster.*