Staff Data Platform Engineer

WEX Inc.; Washington (IL); A distancia; USD 141.000 - 173.000; Jornada completa

Descripción de la vacante

WEX Inc. is seeking a Staff Data Platform Engineer to own our data orchestration and multi-cloud infrastructure. You will manage Airflow clusters, CI/CD pipelines, and containerized workloads across AWS and Azure, driving observability and cost optimization.

You will partner with Data Engineering, Infrastructure, and Operations to deliver scalable data platforms, governance tooling, and robust production systems while mentoring teammates and shaping a 1–2 year platform roadmap.

Formación

• 6+ years of hands-on experience in Data Platform Engineering, DevOps or SRE.
• Deep Airflow production experience and tuning in Kubernetes/Celery environments.
• Automation-first mindset with IaC, GitOps, and CI/CD expertise.
• Strong knowledge of Spark, Snowflake, Azure Synapse and related data tools.

Responsabilidades

• Own Airflow infrastructure design, deployment, scaling, and reliability.
• Drive DevOps using Terraform, Helm, ArgoCD and GitHub Actions for pipelines.
• Build and maintain observability with Grafana, Prometheus and Loki.
• Lead cost optimization and governance tooling across cloud platforms.
• Mentor junior engineers and shape the data platform roadmap.

Conocimientos

Airflow expert

Container orchestration

DevOps automation

Observability & monitoring

Big data stack

Cloud FinOps

Governance tooling

Programming & scripting

Herramientas

Kubernetes

Terraform

Docker

ArgoCD

GitHub Actions

Grafana

Prometheus

Loki

DataHub

Snowflake

Azure Synapse

Apache Kafka

dbt

Descripción del empleo

## Staff Data Platform EngineerApply: Portland, ME: Bay Area, CA: Seattle, WA: San Francisco, CA: Chicago, IL: Full time: Posted Today: R16225This is a remote position; however, the candidate must reside within 30 miles of one of the following locations: Portland, ME; Boston, MA; Chicago, IL; Dallas, TX; San Francisco Bay Area, CA; and Seattle/WA.**About the Team/Role**The Data Platform Engineering Team acts as the backbone of our enterprise data architecture, bridging the gap between Data Engineering, Infrastructure, and Operations. Responsible for architecting, scaling, and maintaining multi-cloud infrastructure across AWS and Azure, the team takes direct ownership of core Apache Airflow orchestration, big data frameworks, and containerized environments to ensure a robust, production-grade platform.We are seeking a Staff Software Engineer (Semantic Foundationa) with 6+ years of hands-on experience to join this team as a core platform engineer and DevOps specialist. In this role, you will take direct ownership of our orchestration and containerization stack while driving key initiatives in CI/CD automation, observability, data governance, and cloud cost optimization. You will bridge the gap between Data Engineering, Infrastructure, and Operations. ****How you’ll make an impact****Infrastructure, Orchestration & DevOps* Airflow Infrastructure Ownership: Design, deploy, scale, and maintain highly available Apache Airflow clusters (using Helm, Kubernetes, and Terraform) to support critical enterprise ETL/ELT workflows.* Infrastructure-as-Code & GitOps: Drive DevOps practices using Terraform, Helm Charts, and ArgoCD to automate platform deployments, manage self-hosted GitHub runners, and enforce GitOps workflows.* CI/CD & Automation: Architect and manage robust CI/CD pipelines utilizing GitHub Actions for seamless deployment of data pipelines, infrastructure components, and DAGs.* Container & Cluster Management: Provision and manage scalable Kubernetes (EKS/AKS) clusters, Docker containers, and underlying cloud infrastructure across AWS (EC2, EMR, S3, VPC) and Azure (Synapse, ADLS).* Observability, Telemetry & Alerting: Build and maintain end-to-end monitoring, logging, and alerting systems using Grafana, Prometheus, Loki, and centralized log management solutions to ensure high platform uptime and reliability.Central Data Platform, Tools & Governance* Big Data Platform Architecture: Build scalable data infrastructure supporting big data processing engines and data warehouses, including Apache Spark, AWS EMR, Snowflake, Azure Synapse, Apache Kafka, and dbt.* Data Lineage & Governance: Deploy and maintain central data discovery and metadata tooling (e.g., DataHub) to facilitate data governance, schema management, and cataloging.* Cost Optimization & FinOps: Actively monitor, audit, and optimize data infrastructure compute and storage costs across AWS and Azure (EC2, EMR, Snowflake queries, Kubernetes nodes).* Developer Experience & Tooling: Build internal tools, CLI utilities, and dynamic workflow templates to improve developer productivity for data engineers, analytics engineers, and data scientists.Technical Leadership & Ownership* Architecture & End-to-End Ownership: Take full technical ownership of data platform modules from architectural design through deployment, production operations, and incident management.* Technical Excellence & Best Practices: Define and enforce high engineering standards for code quality, design patterns, testing, data lineage, and security (access controls, IAM, dynamic schema management).* Strategic Roadmap: Partner with data leads, product managers, and business stakeholders to identify infrastructure gaps, define a 1–2 year data platform roadmap, and prioritize platform initiatives.* Mentorship: Serve as a subject matter expert (SME) on data infrastructure, guiding and mentoring junior and mid-level data platform engineers**.******Experience you’ll bring****Experience & Core Skills* 6+ years of hands-on professional experience in Data Platform Engineering, DevOps, or Site Reliability Engineering (SRE) supporting big data environments.* Airflow Subject Matter Expertise: Deep production experience managing, tuning, dynamic scaling, and troubleshooting Apache Airflow infrastructure (Celery/Kubernetes Executors, DAG parsing performance, dynamic configurations).* Container & Infrastructure Automation: Expert-level skills in Kubernetes, Helm, Terraform, Docker, ArgoCD, and GitHub Actions (including custom runner configurations).* Observability & Monitoring: Proven track record of configuring production alerting, metrics collection, and log aggregation using Grafana, Prometheus, and Loki.* Big Data & Analytics Tech Stack: Deep operational and configuration experience with Spark, AWS EMR, Snowflake, Azure Synapse, dbt, and real-time streaming via Apache Kafka.* Cloud & FinOps: Solid hands-on experience with AWS (EC2, S3, VPC, IAM, EKS) and/or Azure, with a demonstrated history of driving cloud cost optimization.* Governance Tooling: Experience deploying or managing data cataloging tools like DataHub, Amundsen, or similar metadata management platforms.* Programming & Scripting: Strong programming skills in Python, Bash, Go, or SQL.Mindset & Execution* High Ownership: Comfortable taking complex architectural requirements from concept to production-grade deployment in a high-paced environment.* Automation-First Philosophy: Driven to replace manual operational tasks with code, automated tests, automated CI/CD checks, and resilient self-healing infrastructure.The base pay range represents the anticipated low and high end of the pay range for this position. Actual pay rates will vary and will be based on various factors, such as your qualifications, skills, competencies, and proficiency for the role. Base pay is one component of WEX's total compensation package. Most sales positions are eligible for commission under the terms of an applicable plan. Non-sales roles are typically eligible for a quarterly or annual bonus based on their role and applicable plan. WEX's comprehensive and market competitive benefits are designed to support your personal and professional well-being. Benefits include health, dental and vision insurances, retirement savings plan, paid time off, health savings account, flexible spending accounts, life insurance, disability insurance, tuition reimbursement, and more. For more information, check out the \"About Us\" section.Pay Range: $140,600.00 - $173,100.00