Senior Data Engineer: Productionize AI Research (Remote/EST)

Hardshell; United States; Hybrid; USD 149,000 - 182,000; Full time

Job summary

Hardshell is seeking an experienced data systems engineer to move advanced AI security and data science methods from research prototypes into production platforms. You will partner with researchers and engineers, own the technical handoff for demanding components, and help align research priorities with customer and product needs.

Responsibilities include production design, performance tuning, scaling data processing, and cross-team collaboration.

Qualifications

• At least seven years building production data systems or data-intensive software.
• Strong Python skills and experience with distributed data processing.
• Ability to profile and optimize workloads that exceed the capacity of a single machine.
• Hands-on experience with clustering, nearest-neighbor search, or embedding-based approaches at scale.
• Experience with vector databases, retrieval-augmented generation pipelines, or large-scale document processing.
• Strong written communication across research and engineering teams, and U.S. citizenship.

Responsibilities

• Take advanced data science components from prototype through production design, performance tuning, and ongoing maintenance.
• Address scale challenges in clustering and similarity analysis across large document and embedding collections.
• Coordinate progress, readiness, and blockers between research and engineering teams.
• Establish practical handoff standards for reproducibility, testing, and documentation.
• Prioritize engineering work with customer and product requirements in mind.

Skills

Python programming

Distributed data processing

Workload profiling

Clustering / NN search

Vector databases

Communication across teams

U.S. citizenship

Tools

Spark

Vector databases

Job description

Hardshell is seeking an experienced data systems engineer to move advanced AI security and data science methods from research prototypes into production platforms. You will partner with researchers and engineers, own the technical handoff for demanding components, and help align research priorities with customer and product needs.

Responsibilities include production design, performance tuning, scaling data processing, and cross-team collaboration.