Senior Data Engineer, Research Delivery

Hardshell; United States; Hybrid; USD 149,000 - 182,000; Full time

Job summary

Hardshell is seeking an experienced data systems engineer to move advanced AI security and data science methods from research prototypes into production platforms. You will partner with researchers and engineers, own the technical handoff for demanding components, and help align research priorities with customer and product needs.

Responsibilities include production design, performance tuning, scaling data processing, and cross-team collaboration.

Qualifications

• At least seven years building production data systems or data-intensive software.
• Strong Python skills and experience with distributed data processing.
• Ability to profile and optimize workloads that exceed the capacity of a single machine.
• Hands-on experience with clustering, nearest-neighbor search, or embedding-based approaches at scale.
• Experience with vector databases, retrieval-augmented generation pipelines, or large-scale document processing.
• Strong written communication across research and engineering teams, and U.S. citizenship.

Responsibilities

• Take advanced data science components from prototype through production design, performance tuning, and ongoing maintenance.
• Address scale challenges in clustering and similarity analysis across large document and embedding collections.
• Coordinate progress, readiness, and blockers between research and engineering teams.
• Establish practical handoff standards for reproducibility, testing, and documentation.
• Prioritize engineering work with customer and product requirements in mind.

Skills

Python programming

Distributed data processing

Workload profiling

Clustering / NN search

Vector databases

Communication across teams

U.S. citizenship

Tools

Spark

Vector databases

Job description

Role overview

Move complex AI security and data science methods from research prototypes into reliable production systems that can process customer data at scale. You’ll partner with researchers and engineers, own the technical handoff for demanding components, and help align research priorities with customer and product needs.

Responsibilities

• Take advanced data science components from prototype through production design, performance tuning, and ongoing maintenance.
• Address scale challenges in clustering and similarity analysis across large document and embedding collections.
• Coordinate progress, readiness, and blockers between research and engineering teams.
• Establish practical handoff standards for reproducibility, testing, and documentation.
• Prioritize engineering work with customer and product requirements in mind.

Requirements

• At least seven years building production data systems or data-intensive software, including putting research or data science code into production.
• Strong Python skills and experience with distributed data processing.
• Ability to profile and optimize workloads that exceed the capacity of a single machine.
• Hands-on experience with clustering, nearest-neighbor search, or embedding-based approaches at scale.
• Experience with vector databases, retrieval-augmented generation pipelines, or large-scale document processing.
• Strong written communication across research and engineering teams, and U.S. citizenship.

Nice to have

Experience on an applied research team, such as a machine learning research group.

Benefits and work setup

Full-time, hybrid in Charlottesville, Virginia, or remote in the Eastern time zone. Compensation is listed up to $165,000.