Agentic Engineer.

BeyondSOF; Richmond, Northern (VA, KY); Hybrid; USD 120,000 - 160,000; Full time

Job summary

BeyondSOF is seeking an Agentic Engineer to design and develop data pipelines for agentic systems. You will train and fine-tune LLMs, build data architectures, and implement ELT processes to move data into analytics platforms used by data science teams.

You will collaborate with data scientists to preprocess data, integrate AI into applications, and optimize storage for high performance. Strong background in Spark, Azure services, and Python is required.

Qualifications

• Strong data engineering fundamentals.
• Experience with big data frameworks such as Apache Spark and Azure Databricks.
• Ability to train LLMs using structured and unstructured datasets.
• Understanding of graph databases.
• Experience with Azure services including Blob Storage, Data Lakes, Databricks, Azure Machine Learning, Azure OpenAI, and Azure AI tools.
• Proficient in data partitioning and Spark-based schemas.
• Understanding of core ML concepts and algorithms.
• Familiarity with cloud computing concepts and practices.
• Proficiency in Python and AI/ML frameworks.
• Experience with vector databases for retrieval tasks.
• Expertise in integrating with AI agent frameworks.
• Experience with cloud-based AI services, especially Azure AI.
• Proficiency with Git.
• Bachelor’s or Master’s degree in CS/AI/DS or related field.

Responsibilities

• Design and develop data pipelines for agentic systems and robust data flows.
• Train and fine-tune large language models (LLMs).
• Design and build data architecture, including databases and data lakes.
• Develop and manage ELT processes to move data to analytical platforms.
• Implement data pipelines for human-in-the-loop feedback loops.
• Work with vector databases to store and retrieve embeddings.
• Collaborate with data scientists to preprocess data and integrate AI into applications.
• Optimize data storage and retrieval for high performance.
• Conduct statistical analysis to identify trends across multiple sources.

Skills

Data engineering

Apache Spark

Azure Databricks

LLM training

Graph databases

Python

Vector databases

Git

Cloud computing

Machine learning basics

Education

Bachelor’s or Master’s in CS/AI/DS

Tools

Azure Blob Storage

Azure Data Lake

Azure Databricks

Azure OpenAI

Azure Machine Learning

Azure Video Indexer

Azure AI Search

Git

Job description

Interview Mode: Either Phone or In Person

The client is seeking a highly skilled Agentic Engineer to design, develop that solve real-world problems. The ideal candidate will have experience in designing data process to support agentic systems. ensure data quality and facilitating interaction between agents and data.

Responsibilities

• Design and develop data pipelines for agentic systems; develop robust data flows to handle complex interactions between AI agents and data sources.
• Train and fine-tune large language models (LLMs).
• Design and build data architecture, including databases and data lakes, to support various data engineering tasks.
• Develop and manage Extract, Load, Transform (ELT) processes to ensure data is accurately and efficiently moved from source systems to analytical platforms used in data science.
• Implement data pipelines that facilitate feedback loops, allowing human input to improve system performance in human-in-the-loop systems.
• Work with vector databases to store and retrieve embeddings efficiently.
• Collaborate with data scientists and engineers to preprocess data, train models, and integrate AI into applications.
• Optimize data storage and retrieval for high performance.
• Conduct statistical analysis to identify trends and patterns and create data formats from multiple sources.

Qualifications

• Strong data engineering fundamentals.
• Experience with big data frameworks such as Apache Spark and Azure Databricks.
• Ability to train LLMs using structured and unstructured datasets.
• Understanding of graph databases.
• Experience with Azure services including Blob Storage, Data Lakes, Databricks, Azure Machine Learning, Azure Computer Vision, Azure Video Indexer, Azure OpenAI, Azure Media Services, and Azure AI Search.
• Proficient in determining effective data partitioning criteria and implementing partition schemas using Spark.
• Understanding of core machine learning concepts and algorithms.
• Familiarity with cloud computing concepts and practices.
• Strong programming skills in Python and experience with AI/ML frameworks.
• Proficiency in working with vector databases and embedding models for retrieval tasks.
• Expertise in integrating with AI agent frameworks.
• Experience with cloud-based AI services, especially Azure AI.
• Proficiency with version control systems such as Git.
• Bachelor’s or Master’s degree in Computer Science, Artificial Intelligence, Data Science, or a related field.

Required / Desired Amount Experience Understanding the Big data technologies Required 5 Years Experience developing ETL and ELT pipelines Required 5 Years Experience with Spark, GraphDB, Azure Databricks Required 5 Years Experience training LLMs with structured and unstructured data sets Required 4 Years Experience in Data Partitioning and Data conflation Required 3 Years Experience with GIS spatial data Required 3 Years