Starburst Data Federation Lead Engineer

Citi; Jacksonville (FL); On-site; USD 114,000 - 171,000; Full time

Benefits offered by this job

Medical coverage

Dental coverage

Vision coverage

401(k)

Life insurance

Job summary

Citi seeks an Application Development Lead to drive data federation initiatives using Starburst, on-board new catalogs, and implement governance controls. You will lead a global team across time zones—designing scalable data products, ensuring lineage, and enabling BI consumption.

The role emphasizes strong API design, cloud and container platforms, and advanced query optimization for distributed data sources.

Qualifications

• 2+ years of hands-on experience with the Starburst Data Federation and Virtualization platform.
• Deep understanding of Trino architecture, Apache Ranger policies, Starburst Data Products, and the Iceberg table format.
• 6+ years of hands-on experience in Application Development.
• Strong understanding of software architecture principles and distributed system design (Microservices, Event-Driven Architecture).
• Extensive experience with relational (Oracle) and NoSQL databases (MongoDB), including advanced SQL.
• Experience with Big Data technologies (Apache Spark, Hadoop, Kafka) and data warehousing.
• Proven expertise in designing, developing, and securing RESTful APIs and integration patterns.
• Hands-on experience with CI/CD systems (Jenkins, GitHub Actions, Bitbucket Pipelines) for automated deployment.
• Highly experienced with Unix-based operating systems and shell scripting.
• Strong background in cloud platforms (AWS, Azure, GCP), including containerization (Docker) and contributing to Kubernetes orchestration.
• Advanced troubleshooting, debugging, and performance optimization for distributed systems.
• Proficiency in automated testing frameworks (Pytest, unittest) and TDD methodologies.
• Solid understanding of secure coding practices and application security principles (OWASP Top 10).
• Expertise in Git (Bitbucket, GitHub) for advanced branching and pull request workflows.
• Proven ability to lead and mentor a global data engineering and platform operations team across multiple time zones.
• Practical experience with AI tools (Copilot, Devin, Claude/Gemini/GPT) for development workflows.

Responsibilities

• Serve as the primary technical interface between downstream analytical consumers and upstream data source owners.
• Lead the end-to-end technical onboarding of new data catalogs in Starburst using native Starburst connectors (Hive/Iceberg, Oracle, S3).
• Establish operational guidelines with resource and query limits to prevent federated queries from causing performance degradation on critical transactional databases.
• Partner with upstream data stewards to ensure accuracy of business glossaries, data dictionaries, lineage mappings, and data classification tags.
• Implement and govern centralized authorization policies using RBAC and Row-Level Filters in Apache Ranger.
• Define data onboarding standards for Hive/Iceberg table formats, including partitioning schemes, sorting keys, file formats (Parquet/ORC), and statistics collection strategies.
• Validate and tune pushdown capabilities (predicate/projection/join) for newly onboarded data sources to ensure query federation performance.
• Conduct capacity forecasting, analyze query cost efficiency, and implement optimization strategies such as materialization to maximize cluster throughput.
• Partner with enterprise tech enablement teams, data architects, and domain engineers to design, build, and publish standardized, reusable Data Products using Starburst and open data formats.
• Integrate data products with enterprise data catalogs, governance marketplaces, and BI tools, ensuring end-to-end discoverability, automated lineage, and governed access.

Skills

Starburst Data Federation

Trino architecture

Apache Ranger

Iceberg table format

Oracle

MongoDB

Spark/Hadoop/Kafka

RESTful APIs

CI/CD (Jenkins/GitHub Actions/Bitucket

Unix shell

Docker

Kubernetes

AWS/Azure/GCP

Pytest/unittest

OWASP Top 10

Git (Bitbucket/GitHub)

AI tools (Copilot/Claude/Gemini/GPT)

Education

Bachelor’s degree

Master’s degree

Tools

Hive/Iceberg connectors

S3

Jenkins

GitHub Actions

Bitbucket Pipelines

Job description

Citi seeks an Application Development Lead to drive data federation initiatives using Starburst, on-board new catalogs, and implement governance controls. You will lead a global team across time zones—designing scalable data products, ensuring lineage, and enabling BI consumption.

The role emphasizes strong API design, cloud and container platforms, and advanced query optimization for distributed data sources.