Lead Data Engineer - AI & Analytics Platform

Citigroup Inc.; Jersey City (NJ); Hybrid; USD 142,320 - 213,480; Full time

Benefits offered by this job

Hybrid working model

Continuous learning and professional发展

Job summary

Citi in Jersey City, New Jersey, is seeking a Lead Data Engineer for AI & Distributed Analytics to architect and scale enterprise data platforms powering analytics, ML, and Generative AI capabilities.

You will lead a team of data engineers to deliver production-grade, high-performance pipelines bridging raw data to intelligent applications across the enterprise, in a hybrid work model.

Qualifications

• Bachelor's degree in Computer Science, Data Engineering, Information Systems, or a quantitative field.
• 6+ years of professional experience in data engineering, software engineering, or data platform development, including 3+ years in a technical lead or engineering leadership role.
• Expert-level proficiency in Python and SQL, with hands-on experience designing and delivering large-scale, distributed data systems.
• Deep expertise in big data frameworks including Apache Iceberg, Startree/Apache Pinot, and Hive/HDFS, with a strong command of distributed computing principles.
• Hands-on experience building and operating real-time streaming pipelines using Apache Kafka and Apache Flink at high volume and scale.
• Demonstrated ability to design and build data infrastructure specifically supporting machine learning, advanced analytics, or AI applications in production environments.
• Experience working with modern table formats and transactional storage layers such as Delta Lake, Apache Iceberg, or Apache Hudi.
• Solid experience with workflow orchestration tools such as Apache Airflow or equivalent modern orchestrators to manage complex pipeline dependencies.

Responsibilities

• Define and drive the technical vision, architecture, and roadmap for Citi's enterprise Data Lake and Analytics platform, ensuring alignment with cloud-native and hybrid-cloud strategies.
• Lead, mentor, and develop a team of data engineers, setting technical standards and fostering a culture of continuous learning and agile delivery.
• Design and build robust batch and real-time streaming data pipelines using technologies such as Apache Iceberg, Starburst, Startree/Apache Pinot, Apache Kafka, and Apache Flink to support enterprise-scale data processing.
• Architect and implement Feature Stores to standardize feature engineering and ensure consistent, reliable data delivery for model training and real-time ML inference.
• Optimize data storage and query performance across Enterprise Data Lakes and Lakehouses, leveraging platforms such as Delta Lake, Apache Iceberg, and Databricks.
• Implement end-to-end metadata management and data lineage tracking to provide full visibility into how data flows from source systems through to AI models and analytics consumers.
• Establish automated data quality frameworks — including validation rules, reconciliation checks, and compliance controls — to ensure all pipelines meet enterprise security standards and data privacy regulations such as GDPR and CCPA.
• Evaluate and prototype emerging data technologies and frameworks, translating findings into actionable recommendations that keep Citi's data platform at the cutting edge.

Skills

Python

SQL

Kafka

Flink

Apache Iceberg

Delta Lake

Apache Pinot

Airflow

Databricks

Kubernetes

Education

Bachelor's degree in Computer Science or related field

Tools

Docker

Terraform

Job description

Citi in Jersey City, New Jersey, is seeking a Lead Data Engineer for AI & Distributed Analytics to architect and scale enterprise data platforms powering analytics, ML, and Generative AI capabilities.

You will lead a team of data engineers to deliver production-grade, high-performance pipelines bridging raw data to intelligent applications across the enterprise, in a hybrid work model.