Data Engineer — RAG Pipelines, AWS & ElasticSearch

Accenture-Federal-Services-2; Tampa (FL); On-site; USD 119,000 - 162,000; Full time

Job summary

Accenture Federal Services is seeking a Data Engineer to design, develop, and manage large-scale data workflows across structured and unstructured data. The role emphasizes building reliable RAG pipelines, ETL/ELT processes, and deploying scalable data systems in AWS.

The candidate will collaborate with architecture, ML, and application teams to support complex data solutions while ensuring data quality, governance, and security.

Qualifications

• ETL/ELT best practices and data modeling knowledge are required.
• Prefect 2.0 or newer workflow orchestration experience is essential.
• Deep understanding of ElasticSearch indexing, clustering, and search optimization.

Responsibilities

• Design, build, and maintain RAG pipelines, including document ingestion and model retrieval flows.
• Develop and manage structured and unstructured data pipelines in AWS (S3, Lambda, Step Functions, EMR, Glue, ECS/EKS, IAM).
• Implement and operate NiFi flows for high-throughput data ingestion and transformation.
• Orchestrate and schedule workflows using Prefect with reliable error handling and observability.
• Implement indexing, search, and retrieval patterns using ElasticSearch.
• Collaborate with architecture, ML, and application teams to support scalable data solutions.
• Ensure data quality, lineage, governance, and security across pipelines.
• Monitor system performance and troubleshoot distributed data systems.

Skills

ETL/ELT processes

Prefect 2.0

ElasticSearch

Python

Pandas

PySpark

Requests

LLM / Generative AI

Tools

Apache NiFi

Docker

Kubernetes

AWS

CI/CD

OpenSearch

Pinecone

Weaviate

Job description

Accenture Federal Services is seeking a Data Engineer to design, develop, and manage large-scale data workflows across structured and unstructured data. The role emphasizes building reliable RAG pipelines, ETL/ELT processes, and deploying scalable data systems in AWS.

The candidate will collaborate with architecture, ML, and application teams to support complex data solutions while ensuring data quality, governance, and security.