Senior Data Scientist

Socket.dev; New York (NY); On-site; USD 180,000 - 240,000; Full time

Job summary

Socket.dev is seeking a Data Scientist to join as one of our earliest hires, working directly with the founders on key technical decisions from day one. You will implement, evaluate, and improve agentic AI applications and help shape the product roadmap using rigorous benchmarking and real-world deployment.

The role emphasizes building with models, not just building models, and requires deep expertise in compound AI systems, agentic collaboration, and practical tooling (ChatGPT, Cursor, Notebook

Qualifications

• 8+ years experience designing and evaluating LLM-based applications.
• Ability to interpret statistical insights for non-technical audiences (executives).
• Strong programming skills and data analysis capabilities.

Responsibilities

• Evals and Optimization: develop evaluation methods and prompt/context optimization for LLM-based apps.
• Applied Research 0→0 System Design: integrate models and proprietary data into AI-native workflows with benchmarking.
• Exploratory Research – Bold, Big Bets: explore recent advances and drive state-of-the-art opportunities.
• Data-driven investment decisions: apply statistics and ML to sourcing, screening, due diligence, and asset monitoring.

Skills

Experience Building with Models

Uses AI Every Day

Strong Programming & Data Analysis

Biases Towards Showing vs. Telling

Tools

ChatGPT

Cursor

Notebook LM

Claude Code

Job description

The Role

We're looking for Data Scientists who will help implement, evaluate, and improve LLM-based applications (agentic applications). You'll join our team as one of our earliest hires. You will work directly with the founders (ex-Google, ex-Silver Lake) and own key technical decisions from day one.

You'll work on:

• Evals and Optimization You develop innovative ways to evaluate the performance of agentic AI systems, track them over time, and perform prompt/context optimization and adapter tuning where necessary/appropriate to improve the performance of LLM-based applications.
• Applied Research 0→0 System Design You integrate cutting-edge models, proprietary data, and innovative architectures to transform business workflows into AI-native processes. They design, implement, and validate these systems through rigorous benchmarking and real-world deployment.
• Exploratory Research – Bold, Big Bets You critically explore recent advancements drawing from the full breadth of related exploratory efforts to establish company-wide perspectives and opportunities for innovation. You act on these opportunities through bold experimentation, and drive the state of art forward.
• Data-driven investment decisions You understand the work of an investment analyst and can find ways to apply statistics, machine learning, and data science to streamline tasks like sourcing, screening, due diligence, and asset monitoring.

Experience Required:

• Experience Building with Models, not just Building Models We develop intelligent systems using models rather than training or fine-tuning them. Ideal candidates have expertise in compound AI systems, agentic collaboration, and associated techniques (ensembling, ReAct, graph-of-thoughts, etc.).
• Uses AI Every Day Before you can revolutionize someone else's workflow, you need to revolutionize yours. You should be using tools like ChatGPT, Cursor, Notebook LM, and Claude Code to accelerate your workflow.
• Strong Programming and Data Analysis Skills You need to have strong programming skills to be able to prove-out your ideas.
• Biases Towards Showing vs. Telling Our customers want to see the power of AI today vs discuss the most elegant idea that will take 5 years to realize. We believe great Data Scientists come from diverse backgrounds and are unified by deep curiosity, pragmatism, and engineering excellence.

You Have:

• 8+ years experience
• A deep understanding of how to design and evaluate LLM-based applications
• Ability to interpret statistical insights to non-technical or executive audiences
• Strong programming skills

Bonus:

• Experience in financial systems, risk modeling, or decision automation
• Familiarity with ML training and deployment
• Experience with embedded analytics