Senior Data Platform Engineer - AI Infra & Pipelines

Worky; Dallas (TX); On-site; USD 140,000 - 180,000; Full time

Job summary

Ferry International seeks a Sr. Data Platform Engineer to own data pipelines, embeddings, and the AWS infrastructure powering our AI applications. You will build and maintain production-grade data and retrieval systems, ensure reliability, and scale across client needs.

You’ll work across data engineering, cloud infrastructure, DevOps, and AI systems, turning validated ideas into dependable products while maintaining security and cost discipline.

Qualifications

• 5 to 8 years of relevant engineering, data engineering, infrastructure, or related experience.
• Strong Python and SQL skills with hands on experience building data pipelines using PostgreSQL.
• Experience with vector databases or vector search technologies such as pgvector.
• Strong AWS experience, including services such as ECS, Fargate, RDS, and S3.
• Experience with containers, CI/CD, infrastructure as code, and cloud operations.
• Experience building data infrastructure supporting machine learning or LLM applications.
• Hands on experience with embeddings and retrieval systems.
• Working familiarity with LangGraph or the LangChain ecosystem.
• Experience building observability, monitoring, dashboards, and alerting for production systems.
• Strong understanding of system reliability, scalability, security, and cost management.
• Ability to take validated modeling and transform it into production grade systems.
• Strong communication and collaboration skills.

Responsibilities

• Build and maintain production ready data and machine learning pipelines that transform validated modeling into reliable systems.
• Own ingestion, cleaning, embeddings, scoring, and processing across our coaching transcript data.
• Solve the real world data problems that come with a growing system, including mixed storage formats, missing records, historical data, and backfill requirements.
• Own the data corpus and retrieval layer from end to end.
• Modernize embedding infrastructure and consolidate vector storage to improve performance and reliability.
• Ensure retrieval is properly scoped and isolated for each client.
• Own how our AI systems are deployed, operated, monitored, and scaled.
• Build and maintain containerized deployments, CI/CD pipelines, infrastructure as code, environments, observability, alerting, and cost monitoring across AWS.
• Create infrastructure that is reliable, secure, scalable, and cost effective.
• Build the dashboards and monitoring systems that allow the team to understand system performance and identify issues before they become problems.
• Build the durable state, client memory, and retrieval layers that support our AI agent applications.
• Create systems that allow client profiles, commitments, history, and other important information to persist across interactions and over time.
• Make those systems fast, reliable, observable, and scalable.
• You won’t own the overall agent design. You’ll own the infrastructure that makes the agent work consistently in production.
• Turn established evaluation criteria and benchmark data into automated systems that measure AI performance at scale.
• Build automated evaluations that can run through CI and help determine whether new releases are ready for production.
• Develop regression testing and monitoring processes that help the team identify performance issues and maintain quality as the system evolves.
• Treat client and conversation data as sensitive information.
• Build and maintain appropriate systems for retention, access control, encryption, and data security.
• Develop a clean and reliable process for removing client information when requested.
• Help ensure our AI infrastructure is built with security, privacy, and responsible data management in mind.

Skills

Python

SQL

Data pipelines

DevOps

Cloud infrastructure

Observability

Reliability

Collaboration

Tools

PostgreSQL

pgvector

AWS

ECS

Fargate

RDS

S3

Containers

CI/CD

Infrastructure as code

Job description

Ferry International seeks a Sr. Data Platform Engineer to own data pipelines, embeddings, and the AWS infrastructure powering our AI applications. You will build and maintain production-grade data and retrieval systems, ensure reliability, and scale across client needs.

You’ll work across data engineering, cloud infrastructure, DevOps, and AI systems, turning validated ideas into dependable products while maintaining security and cost discipline.