Senior Big Data & PySpark Engineering Lead

Citi; Tampa (FL); On-site; USD 113,840 - 170,760; Full time

Job summary

The Applications Development Technology Lead Analyst at Citi in Tampa, FL leads design and implementation of data pipelines and modern data lakehouse initiatives, aligning with the Technology team. You will drive data engineering, ETL/ELT transformation, and architecture improvements across big data platforms.

The role requires senior experience in data engineering, Hadoop/Spark, SQL, Python, and DevOps practices, with a focus on performance, data integrity, and regulatory compliance within a

Qualifications

• 6+ years of relevant experience in the Financial Service industry
• Experience as Applications Development Manager or senior level in Applications Development
• Experience in Data Engineering, focused on Big Data ecosystems
• Knowledge in Hadoop, YARN, Hive, Impala, Spark, and Spark SQL with Python
• Familiarity with Avro, Parquet, CSV, JSON
• Hands-on experience in writing SQL queries
• Unix-based operating systems and shell scripting
• Experience with Bitbucket, Git etc.
• Big Data Tech Proficiency in Hadoop, Spark, Hive, Kafka, and NoSQL databases (MongoDB, HBase)

Responsibilities

• Design and implement scalable data pipelines for batch and real-time processing
• Lead migration from legacy on-prem SQL to Data Lakehouse in the cloud
• Re-engineer ETL/ELT workflows using Spark and Starburst/Trino
• Ensure data parity and validation across legacy and new systems
• Write high-performance Python code and optimize SQL queries
• Build and maintain CI/CD pipelines for data jobs
• Provide technical mentorship and code reviews

Skills

Python

SQL

Data Modeling

Unix shell scripting

Data engineering

Spark

Kafka

Cloud data lakehouse

Education

Bachelor’s degree / University degree or equivalent experience

Master’s degree preferred

Tools

Git

Bitbucket

Trino

Starburst

Hadoop

Hive

Impala

NoSQL (MongoDB, HBase)

Job description

The Applications Development Technology Lead Analyst at Citi in Tampa, FL leads design and implementation of data pipelines and modern data lakehouse initiatives, aligning with the Technology team. You will drive data engineering, ETL/ELT transformation, and architecture improvements across big data platforms.

The role requires senior experience in data engineering, Hadoop/Spark, SQL, Python, and DevOps practices, with a focus on performance, data integrity, and regulatory compliance within a