Senior Data Engineer: Cloud Data Pipelines & AI Knowledge Base

Solvd, Inc.

United States

Remote

USD 120,000 - 180,000

Full time

8 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Solvd, Inc. is seeking a Senior Data Engineer to build four source connectors and an ingestion pipeline turning raw knowledge content into a governed, permission-tagged corpus in S3 and Bedrock Knowledge Bases.

You will own the ingestion workstream end to end, pairing with a retrieval engineer to ensure metadata, permissions, and lineage are preserved with a tunable sync schedule. The role requires 4+ years in production data pipelines and AWS/Python expertise.

Qualifications

  • 4+ years creating data pipelines in production.
  • 2+ years on AWS with Python.
  • Experience with Microsoft Graph API for SharePoint/Teams.
  • Infrastructure-as-code with Terraform or AWS CDK.
  • CI for pipeline deployment and documentation.

Responsibilities

  • Build the iKNOW connector against Oracle B2C Service knowledge APIs and perform incremental sync.
  • Build connectors on Microsoft Graph for SharePoint/Teams and capture permission metadata.
  • Build the Sitecore connector against content APIs.
  • Implement the ingestion pipeline landing in S3 with metadata enrichment.
  • Produce inventory extracts for content reviews and baselining.
  • Implement retirement handling on next sync.
  • Handle throttling and backoff; enable idempotent re-runs.
  • Instrument every pipeline stage with CloudWatch and write runbooks.
  • Pair with client IT during weeks 7–10 for transition support.

Skills

Data pipelines
AWS
Python
Microsoft Graph API
S3 ingestion
Bedrock KB
Terraform
AWS CDK
CI/CD
Documentation

Tools

OpenSearch
AWS Glue
Step Functions
EventBridge
PDF parsing
Office parsing

Job description

Solvd, Inc. is seeking a Senior Data Engineer to build four source connectors and an ingestion pipeline turning raw knowledge content into a governed, permission-tagged corpus in S3 and Bedrock Knowledge Bases.

You will own the ingestion workstream end to end, pairing with a retrieval engineer to ensure metadata, permissions, and lineage are preserved with a tunable sync schedule. The role requires 4+ years in production data pipelines and AWS/Python expertise.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer - Cloud Ingestion & AI Knowledge Bases
Senior Data Engineer - Cloud Ingestion & AI Knowledge Bases

Solvd • United States

Remote
USD 130,000 - 180,000
Senior Data Engineer — Knowledge Ingestion
Senior Data Engineer — Knowledge Ingestion

Solvd • United States

Remote
USD 130,000 - 180,000
Senior Data Engineer — Knowledge Ingestion
Senior Data Engineer — Knowledge Ingestion

Solvd, Inc. • United States

Remote
USD 120,000 - 180,000
Data Engineer: Real-Time Cloud Pipelines for Risk & Compliance
Data Engineer: Real-Time Cloud Pipelines for Risk & Compliance

Solvd, Inc. • Georgia

On-site
USD 110,000 - 160,000
Senior Data Integration Engineer - AI Pipelines on AWS
Senior Data Integration Engineer - AI Pipelines on AWS

EPAM Systems Inc • United States

Remote
USD 120,000 - 190,000
Senior Data Engineer - Scalable Pipelines & Cloud Analytics
Senior Data Engineer - Scalable Pipelines & Cloud Analytics

Initvalue • Fremont (CA)

Hybrid
USD 130,000 - 190,000
Senior Data Engineer: AI-Driven Cloud Data Pipelines
Senior Data Engineer: AI-Driven Cloud Data Pipelines

Slalom • Los Angeles (CA)

On-site
USD 83,000 - 110,000
401(k) with match
Comprehensive health, dental, & vision
Paid holidays and time off
+2
Senior Data Engineer — Bedrock RAG Retrieval Platform
Senior Data Engineer — Bedrock RAG Retrieval Platform

Solvd, Inc. • United States

Remote
USD 150,000 - 190,000
Senior Data Integration Engineer, AI-Driven Cloud Pipelines
Senior Data Integration Engineer, AI-Driven Cloud Pipelines

EPAM Systems Inc • United States

Remote
USD 120,000 - 180,000
Senior Data Engineer: Cloud Pipelines, Analytics & ML Data
Senior Data Engineer: Cloud Pipelines, Analytics & ML Data

Compunnel Inc. • Charlotte (NC)

On-site
USD 120,000 - 150,000