Senior Data Engineer: Lakehouse & AI/ML Pipelines

Socket.dev; Irving (TX); On-site; USD 107,000 - 161,000; Full time

Job summary

Citi is seeking an Applications Development Senior Programmer Analyst in Irving, TX. The role focuses on building and integrating data pipelines, governance, and Lakehouse architectures across cloud platforms to support analytics and ML workloads.

The position requires strong PySpark, Spark SQL, Databricks, and data governance experience, with Agile/Scrum collaboration and stakeholder engagement.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, Information Technology, or related field.
• Master’s degree is a plus but not required.

Responsibilities

• Develop and maintain scalable ETL/ELT pipelines using PySpark, Spark SQL, and Delta Lake on Databricks.
• Support implementation and management of data platforms across AWS, Azure, or GCP.
• Work with big data technologies to process large datasets (Databricks, Snowflake, Apache Iceberg).
• Optimize Spark workloads through performance tuning and cost-efficient processing.
• Contribute to Lakehouse architecture implementation with data quality and reliability.
• Assist in data federation solutions across multiple data sources (Starburst/Trino).
• Support data governance initiatives including data quality, lineage, metadata, and security policies.
• Ensure data security and governance (RBAC, auditability).
• Troubleshoot and optimize data pipelines and queries.
• Support data modeling for analytics, reporting, and downstream consumption.
• Collaborate with data scientists and ML teams to enable AI/ML pipelines and RAG-based use cases.
• Participate in Agile/Scrum processes.

Skills

PySpark

Spark SQL

Delta Lake

Databricks

AWS

Azure

GCP

Snowflake

Starburst

Trino

Apache Iceberg

Python

RBAC

ETL/ELT

Data governance

Education

Bachelor’s degree in Computer Science, Engineering, IT

Master’s degree preferred

Tools

Databricks

Ab Initio

Job description

Citi is seeking an Applications Development Senior Programmer Analyst in Irving, TX. The role focuses on building and integrating data pipelines, governance, and Lakehouse architectures across cloud platforms to support analytics and ML workloads.

The position requires strong PySpark, Spark SQL, Databricks, and data governance experience, with Agile/Scrum collaboration and stakeholder engagement.