Mid/Senior Data Engineer

Pegasus Knowledge Solutions, Inc.; New York (NY); On-site; USD 120,000 - 190,000; Full time

Job summary

Pegasus Knowledge Solutions, Inc. is seeking an experienced Data Engineer to build pipelines, storage, and logic powering our AI-driven support analytics. You will transform unstructured chats and tickets into clean, structured data our AI agents can reliably use.

You will ingest data from Bliss, Salesforce, Sprinklr, and JIRA; design vector-enabled storage; and deliver a centralized metrics layer. Collaboration with ops, product, and engineering is essential to improve data practices at the

Qualifications

• 5+ years in data engineering with big data tools and cloud data warehouses.
• Strong hands-on Python and PySpark experience.
• Experience using AI tools to accelerate development and parse text.
• Direct experience with vector databases and AI data pipelines.

Responsibilities

• Ingest and clean large volumes of unstructured support data from Bliss, Salesforce, Sprinklr, and JIRA.
• Design storage and retrieval systems including vector databases for AI context.
• Build a centralized metrics layer for AI-driven analytics.
• Collaborate with ops, product, and engineering; improve data practices at the source.

Skills

Big data tools

Python

PySpark

AI tools

Vector databases

Semantic layers

Tools

Pinecone

Milvus

Weaviate

pgvector

Job description

We're transforming our customer support data from static, disconnected reports into a live, AI-powered system. We're looking for an experienced Data Engineer to build the pipelines, storage, and logic that power our new AI agents — turning messy customer chats and tickets into clean, structured information our AI can reliably use.

Job Responsibilities

• Ingest and clean large volumes of unstructured support data from Bliss, Salesforce, Sprinklr, and JIRA
• Design storage and retrieval systems (including vector databases) that give our AI accurate, relevant context
• Build a centralized, reliable metrics layer for AI-driven analytics
• Partner with ops, product, and engineering; push back on poor data practices at the source

Required Skills

• 5+ years in data engineering with big data tools (Hadoop, Hudi, Spark, Presto, Pinot, Flink, Kafka) and cloud data warehouses
• Strong hands‑on Python and PySpark
• Proven use of AI tools (e.g., Claude, Codex) to accelerate development — automating checks, parsing messy text, building data logic layers
• Direct experience with vector databases (Pinecone, Milvus, Weaviate, pgvector) and AI‑feeding data pipelines
• Bonus: experience building metric or semantic layers

If this opportunity aligns with your experience and interests, I'd be happy to discuss it further. Please send your updated resume to [e-mail].