Staff Software Engineer, Data

Jobgether; United States; Remote; USD 176,000 - 230,000; Full time

Benefits offered by this job

Equity program

Health insurance (medical)

HSA/FSA

Retirement plan

Paid parental leave

Flexible time off

Work equipment provided

Open salary discussions

Job summary

Jobgether is seeking a senior data platform engineer to own our data platform end to end, from cloud infrastructure to data models and research pipelines. You will implement lakehouse architectures on AWS, maintain data quality, and build scalable pipelines for multimodal datasets.

Ideal candidates have 6+ years of production data platform experience, strong SQL/Python, and familiarity with Spark, dbt, and Kubernetes-based orchestration.

Qualifications

• At least 6 years building and operating production data platforms.
• Experience integrating SaaS business systems (Stripe, HubSpot, ERP, identity) into a centralized data warehouse.
• Experience with large-scale multimodal datasets and HPC/cloud schedulers (SLURM, Kubernetes).
• Expert-level SQL and Python with strong CI/CD practices.
• Knowledge of AWS, Kubernetes, EKS, Helm, Terraform for production deployments.
• Ability to work autonomously in a small team and own end-to-end production systems.
• Ability to interpret data, communicate limitations, and explain metrics clearly.

Responsibilities

• Own and maintain the data platform end to end, including cloud infrastructure, data lake, and warehouse models.
• Build integrations across billing, CRM, ERP, identity, and telemetry systems to reconcile revenue and usage data.
• Develop data contracts, quality tests, lineage tracking, and data catalog documentation.
• Enforce PII access controls and data classification with Security and Legal teams.
• Process billions of telemetry records and optimize Iceberg tables and partitioning for cost and speed.
• Manage orchestration on Kubernetes/EKS with Helm, Terraform, and CI/CD pipelines.
• Participate in on-call rotations and troubleshoot production data pipelines.
• Deliver operational reporting for Finance and leadership on revenue, credits, and cloud spend.
• Support research teams with sourcing and versioning large training datasets across modalities.
• Build scalable pipelines for media generation and captioning for training data shards.
• Maintain dataset lineage and documentation for reproducibility.
• Lead design reviews, mentor teammates, and develop runbooks for resilience.
• Collaborate with Finance, Marketing, SRE, Security, Product, and Research.

Skills

SQL

Python

Automated testing

CI/CD

Data modeling

Production infrastructure

Kubernetes

AWS

Data governance

Analytical thinking

Tools

Iceberg/Delta Lake on S3

Athena/Trino

dbt

Snowflake/BigQuery

Spark

Airflow/Argo

Kubernetes/EKS

Terraform

Helm

Job description

Accountabilities:

You will own the data platform end to end, from cloud infrastructure and business-system integrations to warehouse modeling and multimodal research pipelines, ensuring data remains reliable, secure, scalable, and accessible.

• Own and maintain the company data platform, including the AWS data lake built with S3, Athena, and Glue, the dbt transformation layer, data ingestion pipelines, and the PostgreSQL serving layer used by dashboards and AI agents.
• Build and maintain integrations across billing, customer relationship management (CRM), enterprise resource planning (ERP), identity, and telemetry systems, ensuring revenue, usage, cloud costs, and customer identities reconcile automatically.
• Develop and maintain warehouse models, dimensional schemas, product event data contracts, data quality tests, lineage tracking, and data catalog documentation.
• Implement and enforce personally identifiable information (PII) access controls and data classification practices, including open and restricted schemas, in collaboration with Security and Legal teams.
• Process large-scale datasets involving billions of telemetry records and terabyte-scale log streams, optimizing Iceberg tables, partitioning strategies, query performance, and infrastructure costs.
• Manage data orchestration and infrastructure on Kubernetes and Amazon EKS, using Helm, Terraform, CI/CD pipelines, monitoring, and alerting to ensure reliable daily operations.
• Participate in the business-hours on-call rotation and troubleshoot failures affecting production data pipelines.
• Deliver operational reporting for Finance and leadership, including revenue by business line, credits and prepaid liabilities, and cloud spending by department.
• Support research teams across audio, video, image, and 3D modalities by sourcing, verifying licenses for, ingesting, and versioning large training datasets.
• Build scalable pipelines for media generation, filtering, deduplication, and captioning, producing training-ready data shards across SLURM clusters and cloud object storage.
• Maintain dataset lineage, quality metrics, and documentation so researchers can reproduce training datasets consistently.
• Lead technical design reviews, mentor teammates, and develop runbooks that improve operational resilience and reduce reliance on individual contributors.
• Collaborate with Finance, Marketing, Site Reliability Engineering (SRE), Security, Product, and Research on cross-functional data initiatives.

Requirements:

The ideal candidate combines strong software engineering discipline, advanced data modeling skills, production infrastructure expertise, and the ability to work independently while taking ownership of complex technical systems.

• At least 6 years of experience building and operating production data platforms, including responsibility for architectural decisions.
• Demonstrated experience integrating SaaS business systems, such as Stripe, HubSpot, ERP platforms, and identity providers, into a centralized data warehouse.
• Production experience preparing large-scale multimodal training datasets and running batch workloads using high-performance computing (HPC) or cloud schedulers such as SLURM, Kubernetes, or equivalent technologies.
• Expert-level SQL and Python skills, supported by strong engineering practices including automated testing, code reviews, and continuous integration.
• Hands‑on experience with data warehouses and data lakes, including data modeling and technologies such as Apache Iceberg or Delta Lake on S3, Athena or Trino, Snowflake or BigQuery, and dbt.
• Experience operating production orchestration systems such as dbt, Airflow, or Argo, and working with distributed processing technologies such as Spark or equivalent frameworks.
• Strong knowledge of AWS, Kubernetes and EKS, Helm, and Terraform, with the ability to troubleshoot issues across data processing jobs and infrastructure deployments.
• Ability to work autonomously in a small team, make sound technical decisions, and take end‑to‑end responsibility for production systems.
• Strong analytical judgment and the ability to interpret data accurately, communicate its limitations, and ensure stakeholders understand what metrics do and do not represent.

Additional strengths that would be valuable include:

• Experience owning and building a data platform from the ground up in a startup environment.
• Experience collaborating with research teams working on generative AI models or handling audio, video, image, and 3D datasets.
• Familiarity with FinOps practices and cloud cost allocation and reporting.
• Knowledge of SOC 2 or similar compliance frameworks governing data access and security.

Benefits:

• Annual base salary: $175,500–$230,000 USD for eligible candidates hired in the United States, depending on relevant experience, skills, and interview performance.
• Equity: Participation in the company equity program for eligible employees.
• Health insurance: Medical coverage for employees and their families, with PPO and HMO options. Selected plans include 100% premium coverage for eligible employees and dependents.
• HSA and FSA accounts: Options to set aside pre‑tax funds for healthcare and other eligible expenses.
• Life insurance: Company‑paid life insurance, with optional additional coverage for employees, spouses, and children.
• Disability coverage: Short‑term and long‑term disability benefits to help protect income in the event of illness or injury.
• Retirement savings: 401(k) plan with a company match of up to 4%.
• Paid parental leave: Support for employees as their families grow.
• Flexible time off: Flexible paid time off and paid company holidays.
• Work equipment: Equipment provided to help employees perform their work effectively.
• Transparent compensation: Open discussions about salary ranges, compensation decisions, and potential leveling throughout the recruitment process.