Principal Data Architect and Manager - Service Special Projects

Socket.dev; Cupertino (CA); On-site; USD 180,000 - 260,000; Full time

Job summary

Socket.dev is building a real-time, petabyte-scale data backbone that ingests, unifies, and serves data for private, personalized experiences across platforms. You will be the Principal Data Architect and Manager, defining architecture and leading the data-engineering team to ship scalable, low-latency pipelines.

You will guide technical vision, drive critical architectural decisions, and mentor senior ICs and managers while delivering on ambitious roadmaps in a privacy-conscious environment.

Qualifications

• MS Degree in Computer Science or related field.
• 12+ years of experience in Data Architecture, Data Engineering, or Platform Engineering.
• Proven experience leading engineers, hiring, performance management and mentorship.
• Track record shipping petabyte-scale, low-latency data platforms in production.
• Deep cloud expertise with AWS S3 and data lakehouse patterns.
• Experience with entity resolution, knowledge graphs, and multimodal data pipelines.
• Experience with LLMs and embedding models in data pipelines and serving.
• Strong streaming knowledge with Kafka or equivalent and Spark/Flink.
• Data modeling for large-scale ingest, retrieval, and analytics; governance and observability.

Responsibilities

• Define and own end-to-end architecture of a real-time, petabyte-scale data backbone.
• Lead and grow the data engineering team, mentor engineers and managers.
• Drive roadmap from design to production, balancing technical vision with delivery.
• Collaborate with cross-functional partners and stakeholders across business units.

Skills

Data Architecture

Data Engineering

Platform Engineering

Cloud (AWS)

Kafka

Spark

Flink

Python

Java

Scala

LLMs

Embeddings

Vector search

Terraform

CI/CD

Observability

Kubernetes

Education

MS Degree in Computer Science

Tools

AWS S3

Kafka

Spark

Flink

Prometheus

Grafana

Datadog

OpenTelemetry

Terraform

Kubernetes

Job description

We're building the large-scale data foundation that powers private, personalized experiences across Apple platforms. Our team designs and operates the systems that ingest, unify, and understand information at massive scale — turning petabytes of data from many sources into a single, high-quality, richly structured representation. This foundation is what intelligent search and on-device experiences rely on, and we build it with an uncompromising bar for data quality, freshness, and privacy. We are looking for a Principal Data Architect and Manager to serve as both the senior technical authority and the people leader for our data platform.

DESCRIPTION

As the Principal Data Architect and Manager on our team, you will serve as both the senior technical authority and the people leader for our data platform. You'll define and own the end-to-end architecture of a real‑time, petabyte‑scale data backbone: from ingestion through a multi‑layered lakehouse to normalized serving layers that power downstream search, ranking, and on‑device experiences. You'll also build, grow, and lead the team of data engineers who bring that architecture to life. This is a hands‑on principal role with multiple facets: you set the technical vision, personally shape the hardest architectural decisions, drive the roadmap through to production, and manage, mentor, and grow the engineers executing against it. Your leverage comes equally from what you design and from the team you build.

MINIMUM QUALIFICATIONS

MS Degree in Computer Science or related degree and 12+ years of experience inData Architecture, Data Engineering, or Platform Engineering, with at least 5years operating in a Principal, Staff, or Lead Manager capacity. Provenexperience leading and managing engineers including hiring, performancemanagement, and technical mentorship of senior ICs and managers. Track record ofshipping petabyte‑scale, low‑latency data platforms in production and operatingthem under real‑world load. Deep cloud expertise: expert‑level proficiency withcloud object storage (e.g., AWS S3) and its architectural nuances for massivedata lakes and lake‑houses. Experience architecting systems for entityresolution, conflation, or knowledge‑graph construction at scale — ideallyinvolving billions of frequently updated entities. Experience designingpipelines that process multimodal data (structured, text, image) and integrateML model inference including LLMs and embedding models: for enrichment andtransformation. Familiarity with LLM/model‑serving infrastructure trade‑offs(inference runtimes, GPU‑backed serving) to inform architectural decisionsStreaming expertise: deep, hands‑on knowledge of Apache Kafka (or comparablebrokers like Kinesis) and complex stream processing (Spark Structured Streaming,Flink, or similar). Data modeling: exceptional ability to design logical andphysical data models for large‑scale ingest, retrieval, and analyticalconsumption — including dimensional modeling and lakehouse patterns. Experiencedefining SLAs, quality metrics, and observability standards for large‑scale dataplatforms, with hands‑on use of monitoring/alerting tooling (e.g.,Prometheus/Grafana, Datadog, or OpenTelemetry‑based tracing). Programming:command of at least one modern data‑pipeline language (Scala, Java, or Python)and strong software engineering fundamentals. Cloud services integration: provenexperience wiring together event notifications, queuing, orchestration, andcompute services into resilient production pipelines. Experience with vectorsearch technologies (e.g., Pinecone, Milvus) and storing/serving embeddings(e.g., pgvector, Milvus, FAISS) Excellent written and verbal communication;proven ability to align engineers, partner teams, and senior leadership frommultiple lines of business around a shared technical direction, with experiencebringing a consumer‑oriented product from inception to production.

PREFERRED QUALIFICATIONS

Experience with embedding storage and retrieval (e.g., pgvector, Milvus, FAISS)and with graph databases (e.g., TigerGraph, Neo4j). Experience deploying,serving, and optimizing LLMs or ML models directly in the production, inferenceruntimes/compilers (ONNX Runtime, TensorRT/TensorRT‑LLM), and serving frameworks(Triton, vLLM, TorchServe or similar). Experience tuning batching, KV‑cache, andGPU utilization for low‑latency, high‑throughput real‑time inference in a datapipeline Experience with data governance tools (e.g., Apache Atlas, AWS GlueCatalog, DataHub). Familiarity with Infrastructure as Code (Terraform, Pulumi)and modern CI/CD practice. Experience designing systems that handle petabytes ofunstructured media data. Working knowledge of data privacy regulations and bestpractice…