Design
Architecture
The installable package is isolated under src/agentic_ontologist/. Everything else is support material.
System overview
flowchart LR
Input["OWL / RDF / JSON-LD"] --> Parser["Parser"]
CLI["CLI or Python API"] --> Workspace["OntologyWorkspace"]
Parser --> Ontology["Ontology"]
Workspace --> Parser
Ontology --> Refine["Refinement"]
Refine --> Graph["ConceptGraph or Hypergraph"]
Graph --> Embed["Graph + text embeddings"]
Embed --> Retrieve["Vector and web retrieval"]
Retrieve --> Agent["Agent runtime"]
Agent --> Validate["Validation + UQ"]
Validate --> Output["Matches, answers, traces"]
classDef boundary fill:#f6f8fa,stroke:#8c959f,color:#1f2328;
classDef core fill:#ddf4ff,stroke:#0969da,color:#1f2328;
class Input,CLI,Output boundary;
class Parser,Workspace,Ontology,Refine,Graph,Embed,Retrieve,Agent,Validate core;
The path-based API and CLI share the same parser, graph, retrieval, and reasoning components. Experiments orchestrate this pipeline without entering the library layer.
Integration pipeline
1. Matching (KROMA)
Exact label and synonym matches, then near names, decided by evidence or a model.
2. Validation
Remove one-to-many and criss-cross conflicts; queue ties for review.
3. Reasoning
Place classes with no match under the target hierarchy, from their matched ancestors.
4. Extraction
Propose the subClassOf and part_of edges an ontology is missing.
The orchestrator in agentic_ontologist.workflow routes every subagent's questions; see Agents.
Package map
| Module | Responsibility |
|---|---|
cli.py and workspace.py | Path-based terminal commands and reusable ontology query sessions |
agents/core/ | Agent orchestration, typed events, and tool execution |
agents/decisions.py | Optional Jev model routing and tool-risk gating, recorded on the existing trace |
agents/runtime/ | Execution traces, a local DuckDB file, or DynamoDB persistence |
agents/ontology_of_thought/ | Conversation memory, thought graphs, optimization, and reconstruction |
agents/tools/ | Shared registry plus ontology, knowledge-graph, and website providers |
algorithms/graph/ | Ontology, concept, graph, hypergraph, predicates, and shared builders |
algorithms/refinement/ | Relation, ontology, offline graph, and online stream refinement |
algorithms/views/ | Hypergraph and multigraph analysis adapters |
parser/ | One format-selecting facade over RDF, JSON, and text adapters; RDF schema declarations stay outside the concept graph |
memory/ | Query cache: hot questions, short-term concept sets, and long-term patterns |
routing/ | Harness of typed questions that replays a cached question or continues into retrieval |
memory_store.py | Saved concept embeddings, separate from the query cache |
embeddings/encoders/ | Text and Node2Vec graph encoders |
embeddings/vector_index/ | Backend-neutral vector search with an exact Torch implementation |
retrieval/ | Structural sampling, web and vector retrieval, and an optional Jev context harness |
functions/similarity/ | Cosine, normalized web distance, hybrid, and cross-graph scoring |
prompting/ | Typed ontology-equivalence prompts and machine-readable answer parsing |
validation/ | Alignment metrics, validation rules, tuple evaluation, pipelines, and ontology linting |
uq/ | Dependency-free agreement scoring and optional UQLM, graph, and ensemble backends |
Algorithm extension points
Graph domain
Extend focused models or use OntologyStructureBuilder to construct new graph representations.
Relation refinement
Configure RelationRefiner with a RuleSet and inspect its structured result.
Ontology refinement
Use OntologyRefiner to refine, rebuild, and preserve ontology metadata.
Offline graph refinement
Use OfflineGraphRefiner for deterministic structural collapse and partition refinement over a complete graph.
Online ontology refinement
OnlineOntologyRefiner follows KROMA Algorithm 2: recompute and sort ranks, split changed classes, apply rank-specific local merges, and defer low-confidence, cyclic, or rank-inconsistent updates. Both refiners accept configuration W's concept_similar predicate.
Vector retrieval
Embedding generation and vector search are separate. Encoders create text and graph vectors, fusion combines them, and a registered vector-index backend performs retrieval.
Encoders
TextEmbedding and GraphEmbedding implement small encoder contracts and can be replaced independently.
Fusion
fuse_embeddings validates dimensions and applies the configured graph/text weighting.
Exact search
TorchExactVectorIndex provides deterministic cosine retrieval for current experiments and tests.
Additional backends
register_vector_index supports optional ANN, DiskANN, and vector-database adapters. Select one through index_backend on graph retrieval.
Retrieval plugins
Use create_web_search with static, dataset, mediawiki, or google-cse. Each provider has visible metadata describing network access and required environment variables. Use register_web_search for a project-specific index. Normalized Google Distance receives the selected provider explicitly and does not hide a network service inside similarity code.
Memory and cache
The Agentic Ontologist keeps two stores for one ontology. Embedding memory holds the concept vectors so the next run can skip embedding. The query cache holds answers so a repeated question can skip the model. They live in different folders and are not interchangeable.
flowchart LR
Ask["Ask"] --> Hot["Hot hash"]
Hot -->|"same question"| Answer["Answer"]
Hot -->|"miss"| Check["Shingle checksum"]
Check -->|"names the concept"| Answer
Check -->|"continue"| Harness["Harness"]
Harness -->|"replay"| Answer
Harness -->|"retrieve"| Retrieve["Retrieve concepts"]
Retrieve --> Replay["Graph replay"]
Replay -->|"same concepts"| Answer
Replay -->|"new concepts"| Model["Model"]
Model --> Short["Short-term trie"]
Short -->|"seen 3 times"| Long["Long-term hash"]
classDef boundary fill:#f6f8fa,stroke:#8c959f,color:#1f2328;
classDef core fill:#ddf4ff,stroke:#0969da,color:#1f2328;
class Ask,Answer,Model boundary;
class Hot,Check,Harness,Retrieve,Replay,Short,Long core;
Replay reads a stored capture. It does not send the SPARQL pattern to an endpoint.
Embedding memory
results/memory stores one folder per ontology: manifest.json and embeddings.pt. agonto memory clear deletes that folder. --no-save-memory keeps the vectors in the current process only.
Hot
hot.json maps a normalized question to its answer. Matching ignores case and extra spaces. The cap is 32 entries. The default policy drops the least frequently read question.
Short-term
short_term.json is a trie of concept ids, the SPARQL pattern, and the answer. Lookup uses the set of concept ids, so order from retrieval does not matter. The cap is 512 nodes.
Long-term
long_term.json is a persistent hash of completed concept sets. A set is copied here after it has been answered 3 times. The cap is 1024 entries. A later question with those concepts replays from here after the short-term trie has dropped them.
One ontology is one directory, results/cache/<sha>/, with hot.json, short_term.json, and long_term.json. The first model answer writes the capture. The same question hits the hot tier before retrieval. After a miss, a checksum of three-word shingles on each remembered answer can name the last concept on that path and skip retrieval. The overlap must be at least 0.75 and lead the next answer by 0.15. A single misspelled word can still match. A question with fewer than 4 shingles continues. That checksum is rebuilt in memory from the concept trie and the long-term patterns. It is not a new cache file. The Harness then asks typed questions about the question the user typed and can replay a cached one. It runs only when the hot cache already has questions. A failed call, or a choice that is not in the cache, continues into retrieval. The default call is SGLang /v1/systemone at http://127.0.0.1:30000 (AGONTO_HARNESS_URL). An empty value keeps the Ollama router, ollama:phi3 (AGONTO_ROUTER_MODEL). Optional Jev routing still chooses among answer models inside the agent and stays separate from this step. A wording that is not replayed hits graph replay when retrieval returns the same concepts. A different concept set still calls the answer model.
Eviction
Each tier drops records when it passes its cap. lfu removes the least frequently read record, then the one touched longest ago. lru removes the record that was read or written longest ago. 2q removes a one-time record before a repeated one. Hot, short-term, and long-term use AGONTO_CACHE_EVICTION, which defaults to lfu. A new policy is a class with choose(entries) registered under a name. A new tier is a new file under agentic_ontologist.memory.cache.
AGONTO_CACHE_DIR moves the folder. AGONTO_HOT_ENTRIES, AGONTO_SHORT_TERM_NODES, and AGONTO_LONG_TERM_ENTRIES set the three caps.
Agent runtime
The public agentic_ontologist.agents API exposes the agent, observers, Ontology-of-Thought models, and supported tools. Internally, the agent emits each event once, the OOT observer owns conversation memory, and the thought graph is derived from that memory for persistence and query optimization. Optional Jev routing and gating add model.routed and tool.blocked to that same event stream. They stay off unless configured, and they do not replace the context harness that selects evidence before prompting.
flowchart TB
User["User query"] --> Router["Optional model router"]
Router --> Core["Agent"]
Core --> Runtime["ModelRuntime"]
Runtime --> LLM["Selected LLM"]
Core --> Gate["Optional tool-risk gate"]
Gate --> Executor["ToolExecutor"]
Executor --> Registry["Tool registry"]
Registry --> OntTools["Ontology providers"]
Registry --> KGTools["Knowledge graph providers"]
Registry --> WebTools["Website providers"]
Core --> Events["Typed event stream"]
Events --> Observer["OOTAgentObserver"]
Observer --> Memory["MemoryTrace"]
Memory --> Thoughts["ThoughtGraph"]
Thoughts --> Optimizer["QueryOptimizer"]
Core --> Persistence["Trace persistence"]
Persistence --> Local["Local JSON"]
Persistence --> Duck["Optional DuckDB"]
Persistence --> Dynamo["Optional DynamoDB"]
classDef external fill:#f6f8fa,stroke:#8c959f,color:#1f2328;
classDef core fill:#ddf4ff,stroke:#0969da,color:#1f2328;
class User,LLM,OntTools,KGTools,WebTools,Local,Duck,Dynamo external;
class Core,Router,Runtime,Gate,Executor,Registry,Events,Observer,Memory,Thoughts,Optimizer,Persistence core;
Core
Agent coordinates the model and delegates registry-backed calls to the tool executor.
Routing
When enabled, Jev chooses local, standard, or careful once per run and records model.routed.
Gating
Configured external lookups can be refused before execution. The refusal is tool.blocked on the same trace.
Events
Observers receive one event stream without depending on the agent implementation.
Memory
MemoryTrace is the canonical record used to derive a ThoughtGraph.
Persistence
AGONTO_TRACE_STORE=duckdb writes each thought graph to results/traces/agentic_ontologist.duckdb. DynamoDB is used when a table is configured.
Agent tools
The Agentic Ontologist intentionally includes only tools used by ontology integration:
- Ontology repository lookup through OLS4, BioPortal, LOV, GitHub RDF, and SPARQL.
- Knowledge-graph lookup through Wikidata, DBpedia, and Neo4j.
- Website context retrieval through Wikipedia and Tavily.
General SPRINT tools for files, geospatial data, PDFs, vision, tabular data, and unrelated workflows are not included.
Extension patterns
- Strategy and Factory: experiment specifications select command-building strategies.
- Template runner: every experiment follows the same build, run, capture, and persist lifecycle.
- Repository: result storage is independent from execution.
- Adapters: external ontology and knowledge sources expose stable tool interfaces.