Senior Data Engineer: Self-Service Data Platform Builder

Intercontinental Exchange (ICE); Atlanta (GA); On-site; USD 120,000 - 165,000; Full time

Job summary

Intercontinental Exchange, Inc. seeks a Senior Data Engineer to lead the Enterprise Architecture data platform effort. You will design and implement scalable, self-service data pipelines with emphasis on data quality and governance, collaborating with stakeholders to support ML/AI workflows.

The role focuses on on-prem data orchestration, performance tuning, and cross-functional leadership to build mission-critical data capabilities across the organization.

Qualifications

• Bachelor's degree required in CS, CE, DS or related field.
• 5+ years of data engineering experience.
• 2+ years on enterprise-scale data platforms.
• Deep expertise with orchestration, performance tuning and operations.
• Strong data transformation and testing experience.
• Experience with stream processing frameworks.
• Proficiency in SQL and Python for pipelines.
• Familiarity with containerized deployments.
• Experience implementing data quality frameworks and CI/CD.
• Ability to collaborate with data scientists and stakeholders.

Responsibilities

• Design, build, and maintain on-premises data orchestration platform using best-in-breed open source tools.
• Create self-service capabilities for teams to build and deploy data pipelines.
• Implement robust data quality testing frameworks to ensure data integrity.
• Establish data engineering best practices, including version control, CI/CD, and automated testing.
• Collaborate with ML/AI teams to build scalable feature engineering pipelines for batch and real-time processing.
• Develop reusable patterns for common data integration scenarios across the org.
• Work with infrastructure teams to optimize Kubernetes-based data platform for performance and reliability.
• Mentor junior engineers and advocate for engineering excellence in data practices.

Skills

Workflow orchestration

Performance optimization

Operational management

Data transformation

Testing frameworks

Deployment strategies

Stream processing

SQL

Python

Containerization

Data quality

Cross-functional collaboration

Education

Bachelor's degree in CS/CE/DS

Tools

Kubernetes

CI/CD

Job description

Intercontinental Exchange, Inc. seeks a Senior Data Engineer to lead the Enterprise Architecture data platform effort. You will design and implement scalable, self-service data pipelines with emphasis on data quality and governance, collaborating with stakeholders to support ML/AI workflows.

The role focuses on on-prem data orchestration, performance tuning, and cross-functional leadership to build mission-critical data capabilities across the organization.