Content Pipeline Engineer

Black Canyon Consulting; Washington (District of Columbia); Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

Medical, dental and vision coverage

401k plan with employer contribution

Paid holidays & vacation

Tuition reimbursement

Job summary

Black Canyon Consulting seeks a Content Pipeline Engineer to design and implement reusable pipelines for extracting, structuring, indexing, and retrieving information from permitting documents, maps, and other unstructured content.

You will develop OCR and multimodal processing, vector indexing, semantic retrieval, and agentic workflows, handling PDFs, drawings, plats, and legal descriptions while ensuring provenance and scalable data pipelines in a remote setting.

Qualifications

• BS/BA in CS, Software Engineering, Data Science, or related field.
• 5+ years of software engineering, data engineering, search, document-processing, or related experience.
• Strong Python development experience and production-grade data pipelines.
• Experience with PDFs/images OCR and document-extraction technologies.
• Experience with document stores, search engines, or vector databases and semantic retrieval.
• Experience with LLM-enabled or agentic document-processing workflows.

Responsibilities

• Design reusable ingestion and processing pipelines for PDFs, scanned documents, images, forms, maps, and other unstructured content.
• Implement OCR and multimodal document-processing workflows for standard and map-related materials.
• Extract text, tables, metadata, identifiers, and domain-specific attributes from sources.
• Develop document parsing, classification, normalization, chunking, and metadata-enrichment processes.
• Maintain linkage and provenance between extracted information, source documents, and related records.
• Build document and vector indexes using OpenSearch or comparable technologies.
• Implement embeddings, semantic search, and retrieval pipelines.
• Develop APIs enabling downstream applications to search and retrieve content.
• Establish agentic workflows for classification, extraction strategies, validation, and QA.

Skills

Python development

OCR

Document processing

LLM-enabled workflows

Data pipelines

Geospatial data handling

Education

Bachelor's degree in Computer Science or related field

Tools

OpenSearch

LangChain

LangGraph

AWS

Job description

We are seeking aContent Pipeline Engineer to design and implement reusable pipelines for extracting, structuring, indexing, and retrieving information from permitting documents, maps, and other unstructured content.

The Content Pipeline Engineer will build document-processing capabilities spanning OCR, document parsing, multimodal extraction, metadata enrichment, vector indexing, semantic retrieval, and agentic workflows. The role will support documents such as PDFs, scanned forms, well files, plats, survey drawings, maps, and legal descriptions and make their content available to downstream applications and AI services.

This position is remote.

Duties & Responsibilities

• Design and implement reusable ingestion and processing pipelines for PDFs, scanned documents, images, forms, maps, and other unstructured content.


• Implement OCR and multimodal document-processing workflows for both standard documents and map-related materials.


• Extract text, tables, metadata, identifiers, and domain-specific attributes from source documents.


• Support extraction of information from plats and survey drawings, including legal descriptions, labels, bearings, distances, and other spatially relevant information.


• Develop document parsing, classification, normalization, chunking, and metadata-enrichment processes.


• Maintain linkage and provenance between extracted information, source documents, originating systems, and related business records.


• Build document and vector indexes using OpenSearch or comparable technologies.


• Implement embeddings, semantic search, and retrieval pipelines.


• Develop APIs and reusable interfaces enabling downstream applications and AI services to search and retrieve document content.


• Implement agentic workflows for activities such as document classification, extraction strategy selection, validation, exception handling, and quality assurance.


• Establish confidence thresholds and human-review processes for uncertain or ambiguous extractions.


• Work with GIS engineers to connect map and document extraction results with georeferenced and vectorized geometry.


• Work with Data Scientists to create evaluation datasets and measure OCR, extraction, retrieval, and classification quality.


• Build automated testing, monitoring, and observability for document-processing pipelines.


• Optimize processing for accuracy, throughput, repeatability, and cost.

Educational Requirements

• Bachelor's degree in Computer Science, Software Engineering, Data Science, Information Systems, or a related technical field.

Requirements

• 5+ years of software engineering, data engineering, search, document-processing, or related experience.


• Strong Python development experience and experience building production-grade data or content-processing pipelines.


• Hands-on experience processing PDFs, images, scanned documents, and semi-structured content using OCR and document-extraction technologies.


• Experience with document stores, search engines, or vector databases, including embeddings and semantic retrieval.


• Experience with document parsing, classification, chunking, metadata enrichment, and preservation of source provenance.


• Experience with LLM-enabled or agentic document-processing workflows, including extraction, validation, routing, or exception handling.


• Experience building robust pipelines with automated testing, logging, monitoring, error handling, and quality controls.


• Experience evaluating OCR and extraction quality using measurable criteria and confidence thresholds.


• Strong problem-solving skills for working with complex and inconsistent document formats.

Bonus Skills

• Experience with OpenSearch.


• Experience with AWS document-processing, AI, or machine-learning services.


• Experience with agentic frameworks such as LangChain, LangGraph, or similar orchestration frameworks.


• Experience with multimodal models, vision-language models, or LLM-based document extraction.


• Experience building RAG or enterprise semantic-search solutions.


• Experience with maps, engineering drawings, plats, survey documents, or legal descriptions.


• Experience extracting bearings, distances, coordinates, or other geospatial information from documents.


• Experience working in the U.S. Federal Government space.

Experience with permitting, environmental, energy, land-management, or regulatory documents.

Benefits and Salary

We attract the best people in the business with our competitive benefits package that includes medical, dental and vision coverage, 401k plan with employer contribution, paid holidays, vacation, and tuition reimbursement.

Voluntary Self-Identification

For government reporting purposes, we ask candidates to respond to the below self-identification survey.Completion of the form is entirely voluntary. Whatever your decision, it will not be considered in the hiringprocess or thereafter. Any information that you do provide will be recorded and maintained in aconfidential file.

As set forth in Black Canyon Consulting’s Equal Employment Opportunity policy,we do not discriminate on the basis of any protected group status under any applicable law.

If you believe you belong to any of the categories of protected veterans listed below, please indicate by making the appropriate selection.As a government contractor subject to the Vietnam Era Veterans Readjustment Assistance Act (VEVRAA), we request this information in order to measurethe effectiveness of the outreach and positive recruitment efforts we undertake pursuant to VEVRAA. Classification of protected categoriesis as follows:

A \"disabled veteran\" is one of the following: a veteran of the U.S. military, ground, naval or air service who is entitled to compensation (or who but for the receipt of military retired pay would be entitled to compensation) under laws administered by the Secretary of Veterans Affairs; or a person who was discharged or released from active duty because of a service-connected disability.

A \"recently separated veteran\" means any veteran during the three-year period beginning on the date of such veteran's discharge or release from active duty in the U.S. military, ground, naval, or air service.

An \"active duty wartime or campaign badge veteran\" means a veteran who served on active duty in the U.S. military, ground, naval or air service during a war, or in a campaign or expedition for which a campaign badge has been authorized under the laws administered by the Department of Defense.

An \"Armed forces service medal veteran\" means a veteran who, while serving on active duty in the U.S. military, ground, naval or air service, participated in a United States military operation for which an Armed Forces service medal was awarded pursuant to Executive Order 12985.