Lead Agentic Data Engineer

Apex Systems; Richmond (VA); On-site; USD 80,000 - 120,000; Full time

Job summary

An innovative firm is on the lookout for an Agentic Data Engineer to design and develop cutting-edge data pipelines utilizing agentic AI. This role involves mentoring AI engineers, leading impactful projects, and implementing AI strategies that align with the organization's vision. You will work on advanced data processes, ensuring data quality while facilitating interactions between AI agents and data. If you are passionate about leveraging AI to solve real-world challenges and enjoy working in a collaborative environment, this position offers a unique opportunity to make a significant impact in the field of data engineering.

Qualifications

• Experience training LLMs with structured and unstructured data.
• Strong programming skills in Python with AI/ML frameworks.
• Proven leadership in managing AI projects and mentoring engineers.

Responsibilities

• Guide and mentor AI engineers, enhancing their skills.
• Lead AI projects, ensuring timely and actionable outcomes.
• Design and develop data pipelines for agentic systems.

Skills

Python Programming

AI/ML Frameworks

Leadership

Problem-Solving

Communication Skills

Critical Evaluation

Flexibility in Learning

Education

Bachelor’s degree in Computer Science

Master’s degree in AI or Data Science

Tools

Azure Blob Storage

Azure Data Lakes

Azure Databricks

Spark

Vector Databases

GIS Spatial Data

Job description

Apex Systems is seeking an Agentic Data Engineer to design, develop, and deploy data pipelines that leverage agentic AI to solve real-world problems.

The ideal candidate will have experience in designing data processes to support agentic systems, ensure data quality, and facilitate interaction between agents and data.

Responsibilities:

1. Guiding and mentoring AI engineers, helping them develop their skills and knowledge in the field.
2. Leading and managing AI projects, ensuring they stay on track, meet deadlines, and the findings are actionable and relevant.
3. Contributing to the creation and implementation of AI strategies that align with the organization's goals and objectives.
4. Designing and developing data pipelines for agentic systems, developing robust data flows to handle complex interactions between AI agents and data sources.
5. Using advanced mathematical modeling, statistical analysis, and optimization techniques to gather and analyze data, identifying problems and developing solutions to improve efficiency in prompts.
6. Training and fine-tuning large language models and designing and building the data architecture, including databases, data warehouses, and data lakes, to support various data engineering tasks.
7. Developing and managing Extract, Load, Transform (ELT) processes to ensure data is accurately and efficiently moved from source systems to analytical platforms used in data science.
8. Implementing data pipelines that facilitate feedback loops, allowing human input to improve system performance in human-in-the-loop systems.
9. Working with vector databases to store and retrieve embeddings efficiently.
10. Collaborating with data scientists and engineers to preprocess data, train models, and integrate AI into applications.
11. Optimizing data storage and retrieval with high performance.

Qualifications:

1. Training LLMs with structured and unstructured data sets.
2. Understanding of Graph DB.
3. Experience with Azure Blob Storage, Azure Data Lakes, Azure Databricks.
4. Utilizing data storage systems with Spark to implement partition schemes.
5. Understanding core machine learning concepts and algorithms.
6. Familiarity with cloud computing skills.
7. Strong programming skills in Python and experience with AI/ML frameworks.
8. Proficiency in vector databases and embedding models for retrieval tasks.
9. Expertise in integrating with AI agent frameworks.
10. Experience with cloud AI services (Azure AI).
11. Experience with GIS spatial data.
12. Strong leadership, excellent problem-solving, and communication skills.
13. Proven experience in leading projects and teams, including the mentorship of AI engineers.
14. The ability to engage in critical evaluation of information, hypothesis testing, and scenario analysis.
15. Flexibility in learning and adopting new technologies, methodologies, and tools to stay at the forefront of AI trends.
16. Experience with Data Domains developing an AI Composite Agentic Solution designed to identify and analyze data models, connect and correlate information to validate hypotheses, forecast, predict and recommend potential strategies, and conduct what-if analysis.
17. Bachelor’s or master’s degree in computer science, AI, Data Science, or a related field.