Senior Data Engineer - Cloud Data Ingestion & AI Pipelines

Solvd, Inc.; United States; Remote; USD 120,000 - 170,000; Full time

Job summary

Solvd, Inc. is seeking a Senior Data Engineer to design and implement end-to-end ingestion pipelines that move content from multiple sources into a governed, metadata-rich corpus in S3 and Bedrock Knowledge Bases.

You will build connectors for Oracle B2C Service, Microsoft Graph, and Sitecore, while ensuring data freshness, permissions, and lineage across syncs. You’ll work with the Solution Architect to define architecture, handle throttling and retries, and deliver runnable pipelines with

Qualifications

• 4+ years building data pipelines in production.
• 2+ years on AWS with Python.
• Hands-on with Microsoft Graph API for SharePoint and Teams content or equivalent.
• Experience landing unstructured content into S3 and preparing it for search or vector indexing.
• IaC experience with Terraform or AWS CDK and CI for pipeline deployment.

Responsibilities

• Build the iKNOW connector against Oracle B2C Service knowledge APIs with full inventory extraction and incremental sync.
• Build SharePoint, OneDrive, and Teams connectors on Microsoft Graph with delta queries and permission metadata capture.
• Build the Sitecore connector against its content APIs for public web content in scope.
• Implement the ingestion pipeline: landing in S3, per-source prefixes, text extraction, deduplication, taxonomy enrichment, Bedrock data source sync.
• Produce inventory extracts for content disposition reviews and re-baselining.
• Implement retirement handling: retired/deleted items removed on next sync.

Skills

Data pipelines

AWS

Python

Microsoft Graph API

Unstructured content ingestion

Bedrock/OpenSearch

Terraform/AWS CDK

Documentation

Tools

Terraform

AWS CDK

OpenSearch

Microsoft Graph

Job description

Solvd, Inc. is seeking a Senior Data Engineer to design and implement end-to-end ingestion pipelines that move content from multiple sources into a governed, metadata-rich corpus in S3 and Bedrock Knowledge Bases.

You will build connectors for Oracle B2C Service, Microsoft Graph, and Sitecore, while ensuring data freshness, permissions, and lineage across syncs. You’ll work with the Solution Architect to define architecture, handle throttling and retries, and deliver runnable pipelines with