Data Ingestion & Infra Engineer for AI Datasets

United States Digital Space LLC; Manchester; On-site; GBP 70,000 - 110,000; Full time

Benefits offered by this job

Competitive salaries

Asynchronous culture

Hands-off management

Life-changing product impact

Remote-friendly company culture

Global distributed team

Impactful learning differences focus

Job summary

United States Digital Space LLC is seeking a Software Engineer to join our data-focused AI team. You will own data collection and ingestion, building high-quality datasets at petabyte scale, and collaborating with scientists to optimize cost, throughput, and quality for next-generation models.

You will operate cloud infrastructure (GCP) and Terraform-based workflows, contributing to our distributed, fast-paced environment and impactful, life-changing products that help people with learning

Qualifications

• BS/MS/PhD in Computer Science or a related field.
• 5+ years of industry experience in software development.
• Proficiency with bash/Python scripting in Linux environments.
• Proficiency in Docker and Infrastructure-as-Code concepts and professional experience with at least one major Cloud Provider (we use GCP).
• Experience with web crawlers, large-scale data processing workflows is a plus.

Responsibilities

• Be scrappy to find new sources of audio data and bring it into our ingestion pipeline.
• Operate and extend the cloud infrastructure for our ingestion pipeline, currently running on GCP and managed with Terraform.
• Collaborate closely with our Scientists to shift the cost/throughput/quality frontier, delivering richer data at bigger scale and lower cost to power our next-generation models.
• Collaborate with others on the AI Team and the company Leadership to craft the AI Team's dataset roadmap to power the company's next-generation consumer and enterprise products.

Skills

Bash/Python

Docker

IaC

GCP

Data ingestion

Data pipelines

Cloud

Communication

Education

BS/MS/PhD in Computer Science or related field

Tools

Terraform

Job description

United States Digital Space LLC is seeking a Software Engineer to join our data-focused AI team. You will own data collection and ingestion, building high-quality datasets at petabyte scale, and collaborating with scientists to optimize cost, throughput, and quality for next-generation models.

You will operate cloud infrastructure (GCP) and Terraform-based workflows, contributing to our distributed, fast-paced environment and impactful, life-changing products that help people with learning