Solution Architect - Agentic AI Data

Theron Solutions; United States; On-site; USD 110,000 - 150,000; Full time

Job summary

A leading consulting firm in the United States is seeking a Mid-Senior Level Data Architect. The role involves designing data architectures for AI agents, ensuring high data quality, and collaborating closely with AI developers. Ideal candidates will have strong expertise in knowledge management, data modeling, and will uphold ethical data practices. This is a full-time position within a dynamic work environment.

Qualifications

• Experience with data architecture for AI agents.
• Strong understanding of information retrieval concepts.
• Proficiency in data modeling for knowledge representation.

Responsibilities

• Design the data architecture for AI agents.
• Collaborate with AI engineers to align data with model behavior.
• Ensure privacy and ethical handling of data.

Skills

Knowledge Management & Retrieval Expertise

Data Modeling & Ontologies

NLP and Unstructured Data Handling

Vector Databases & Semantic Search

Graph Database Knowledge

Data Quality & Curation Mindset

Analytical Thinking

Collaboration & Communication

Attention to Detail

Adaptability & Continuous Learning

Domain Knowledge Flexibility

Experience with AI Projects

Ethical and Secure Data Handling

Tools

Elasticsearch

Python

SQL/NoSQL Databases

Cloud Services for AI Data

NLP Toolkits

Job description

What You Would Be Doing

• Knowledge Base Architecture: Design the data and knowledge base architecture for AI agents, including choosing the right type of storage for different data needs and defining how data is organized (schemas, ontologies) for effective querying.
• Data Modeling for AI Agents: Develop data models representing domain knowledge required by AI agents, facilitating the kinds of questions or tasks the agent will handle.
• Memory and State Management: Architect how AI agents store and update their state or memory, ensuring consistency and isolation of context between different sessions or agent instances.
• Retrieval-Augmented Generation (RAG) Pipelines: Plan and design the pipeline for retrieval-augmented generation, outlining how new data gets processed and included to keep the agent’s knowledge current.
• Multi-Modal Data Integration: Design an integrated data approach for agents working with multiple data modalities, allowing retrieval from various sources.
• Data Pipeline Coordination: Work with Data Engineering teams to establish pipelines that feed the agent’s data stores, ensuring automated processes populate and refresh the data architecture.
• Performance Optimization for Retrieval: Ensure efficient data access for agents by analyzing query patterns and optimizing storage or retrieval mechanisms.
• Data Quality and Curation: Develop approaches for curating and filtering data, maintaining a high-quality knowledge corpus.
• Industry-Specific Knowledge Frameworks: Tailor knowledge architecture to industry needs, understanding domain data and shaping it into a form usable by AI agents.
• Metadata and Ontologies: Define metadata standards and ontologies for the agent’s knowledge, improving the agent’s ability to fetch relevant info and do basic reasoning.
• Collaboration with AI Developers: Work closely with AI engineers and data scientists to align data usage with model behavior, adjusting data design based on AI performance metrics.
• Documentation and Governance: Document the data architecture for AI agents and implement governance around the knowledge base.
• Ethical Data Use & Privacy: Ensure the design respects privacy and ethical considerations, handling removal requests and segregating indexes as needed.

What Skills Are Expected

• Knowledge Management & Retrieval Expertise: Strong understanding of information retrieval concepts, search engines, and knowledge management systems.
• Data Modeling & Ontologies: Skills in data modeling for knowledge representation, including experience with knowledge graphs and ontologies.
• NLP and Unstructured Data Handling: Familiarity with natural language processing techniques for processing unstructured text.
• Vector Databases & Semantic Search: Experience with semantic search and vector databases.
• Graph Database Knowledge: Knowledge of graph databases and query languages.
• Data Quality & Curation Mindset: Keen sense for data quality issues and designing measures to maintain quality.
• Analytical Thinking: Strong analytical and problem-solving skills around data.
• Collaboration & Communication: Ability to communicate with both technical and non-technical stakeholders.
• Attention to Detail: Meticulous oversight ensuring metadata is correctly applied and sources are well-documented.
• Adaptability & Continuous Learning: Eagerness to keep up with research and emerging tools.
• Domain Knowledge Flexibility: Ability to quickly learn the basics of various domains.
• Experience with AI Projects: Prior experience working on AI or advanced analytics projects from the data side.
• Ethical and Secure Data Handling: Knowledge of privacy laws and ethical guidelines, ensuring secure data handling.

Key Technology Capabilities

• Search Engines & Indexing: Proficiency with search technologies like Elasticsearch or Solr.
• Vector Search & LLM Integration: Hands-on experience with vector databases or libraries for embedding-based search.
• Graph Databases: Comfortable with graph databases and query languages.
• Data Pipelines & ETL Tools: Knowledge of data pipeline tools for ingesting and updating knowledge stores.
• NLP Toolkits: Familiarity with NLP libraries for text preprocessing tasks.
• Database Management: Skills in managing and querying SQL/NoSQL databases.
• Cloud Services for AI Data: Familiarity with cloud services geared towards search and AI data.
• Scripting and Programming: Strong scripting skills in Python or other relevant languages.
• API & Microservices: Ability to design or use APIs for data retrieval.
• Caching & In-Memory Stores: Knowledge of caching mechanisms to speed up frequent queries.
• Data Analytics Tools: Ability to use analytics or BI tools to examine data.
• Quality and Annotation Tools: Familiarity with tools for data annotation or curation.
• Monitoring & Logging: Setting up monitoring for data pipelines and knowledge base health.
• Security & Access Control: Use of security features in data stores.
• Collaboration Tools: Proficiency with documentation and task tracking tools.
• AI/ML Basics: Understanding how models use data.
• Domain-Specific Platforms: Knowledge of domain-specific data platforms.

EOE: Our client is an Equal Opportunity Employer and Prohibits Discrimination and Harassment of Any Kind: We are committed to the principle of equal employment opportunity for all employees and to providing employees with a work environment free of discrimination and harassment. All employment decisions at our client are based on business needs, job requirements, and individual qualifications, without regard to race, color, religion or belief, national, social or ethnic origin, sex (including pregnancy), age, physical, mental or sensory disability, HIV Status, sexual orientation, gender identity and/or expression, marital, civil union or domestic partnership status, past or present military service, family medical history or genetic information, family or parental status, or any other status protected by the laws or regulations in the locations where we operate. We will not tolerate discrimination or harassment based on any of these characteristics.

Seniority level: Mid-Senior level

Employment type: Full-time

Job function: Consulting

Industries: Business Consulting and Services

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.