Data Platform Engineer for AI & Multimodal Pipelines

Jobgether; Oregon; Hybride; USD 176 000 - 230 000; Plein temps

Avantages offerts par ce poste

Health insurance

401(k) plan

Equity

Paid parental leave

Flexible time off

Equipment provided

Résumé du poste

Jobgether, on behalf of a partner company, seeks a Staff Software Engineer, Data based in the United States. You will own and evolve a comprehensive data platform supporting business operations and AI research, designing scalable data infrastructure and building large-scale pipelines for multimodal training datasets.

Collaborate with Finance, Product, Security, Infrastructure, Marketing, and Research to ensure reliable data flows, governance, and cost efficiency; mentor teammates and lead design

Qualifications

• 6+ years building and operating production data platforms with architectural responsibility.
• Experience integrating SaaS business systems (Stripe, HubSpot, ERP, identity providers) into a central warehouse.
• Experience with large multimodal training datasets and HPC/cloud schedulers (SLURM, Kubernetes).
• Expert-level SQL and Python with strong CI/CD and testing practices.
• Hands-on with data warehouses and lakes (Iceberg/Delta on S3, Athena/Trino, Snowflake/BigQuery) and dbt.
• Experience operating orchestration systems (dbt, Airflow, Argo) and Spark-like frameworks.
• Strong AWS, Kubernetes/EKS, Helm, Terraform knowledge; troubleshoot data/infra issues.
• Autonomous, end-to-end ownership in a small team.
• Analytical judgment and clear communication of data limitations.

Responsabilités

• Own and maintain the company data platform, including the AWS data lake (S3, Athena, Glue), dbt, pipelines, and PostgreSQL serving layer for dashboards and AI agents.
• Build integrations across billing, CRM, ERP, identity, and telemetry systems to reconcile revenue, usage, costs, and customer identities.
• Develop warehouse models, data contracts, quality tests, lineage tracking, and data catalog documentation.
• Enforce PII access controls and data classification with Security and Legal teams.
• Process terabyte-scale logs and billions of telemetry records; optimize Iceberg, partitioning, and costs.
• Manage data orchestration on Kubernetes/EKS with Helm, Terraform, CI/CD, monitoring, and alerts.
• Participate in on-call rotation; troubleshoot production data pipelines.
• Deliver operational reporting for Finance and leadership (revenue by line, credits, cloud spend).
• Support research teams across modalities by sourcing/licensing, ingesting, and versioning large training datasets.
• Build pipelines for media generation, filtering, deduplication, and training shards with SLURM and cloud storage.
• Maintain dataset lineage and docs for reproducibility; lead design reviews and mentor teammates.

Connaissances

SQL

Python

AWS

Kubernetes

dbt

Airflow

Spark

Data modeling

CI/CD

ETL

Outils

Apache Iceberg

Delta Lake

S3

Athena

Glue

dbt

Snowflake

BigQuery

Kubernetes

Airflow

Argo

Terraform

SLURM

Description du poste

Jobgether, on behalf of a partner company, seeks a Staff Software Engineer, Data based in the United States. You will own and evolve a comprehensive data platform supporting business operations and AI research, designing scalable data infrastructure and building large-scale pipelines for multimodal training datasets.

Collaborate with Finance, Product, Security, Infrastructure, Marketing, and Research to ensure reliable data flows, governance, and cost efficiency; mentor teammates and lead design