Agentic Ontologist

Design

Architecture

The installable package is isolated under src/agentic_ontologist/. Everything else is support material.

System overview

flowchart LR
    Input["OWL / RDF / JSON-LD"] --> Parser["Parser"]
    CLI["CLI or Python API"] --> Workspace["OntologyWorkspace"]
    Parser --> Ontology["Ontology"]
    Workspace --> Parser
    Ontology --> Refine["Refinement"]
    Refine --> Graph["ConceptGraph or Hypergraph"]
    Graph --> Embed["Graph + text embeddings"]
    Embed --> Retrieve["Vector and web retrieval"]
    Retrieve --> Agent["Agent runtime"]
    Agent --> Validate["Validation + UQ"]
    Validate --> Output["Matches, answers, traces"]

    classDef boundary fill:#f6f8fa,stroke:#8c959f,color:#1f2328;
    classDef core fill:#ddf4ff,stroke:#0969da,color:#1f2328;
    class Input,CLI,Output boundary;
    class Parser,Workspace,Ontology,Refine,Graph,Embed,Retrieve,Agent,Validate core;
      

The path-based API and CLI share the same parser, graph, retrieval, and reasoning components. Experiments orchestrate this pipeline without entering the library layer.

Integration pipeline

1. Matching (KROMA)

Exact label and synonym matches, then near names, decided by evidence or a model.

2. Validation

Remove one-to-many and criss-cross conflicts; queue ties for review.

3. Reasoning

Place classes with no match under the target hierarchy, from their matched ancestors.

4. Extraction

Propose the subClassOf and part_of edges an ontology is missing.

The orchestrator in agentic_ontologist.workflow routes every subagent's questions; see Agents.

Package map

ModuleResponsibility
cli.py and workspace.pyPath-based terminal commands and reusable ontology query sessions
agents/core/Agent orchestration, typed events, and tool execution
agents/decisions.pyOptional Jev model routing and tool-risk gating, recorded on the existing trace
agents/runtime/Execution traces, a local DuckDB file, or DynamoDB persistence
agents/ontology_of_thought/Conversation memory, thought graphs, optimization, and reconstruction
agents/tools/Shared registry plus ontology, knowledge-graph, and website providers
algorithms/graph/Ontology, concept, graph, hypergraph, predicates, and shared builders
algorithms/refinement/Relation, ontology, offline graph, and online stream refinement
algorithms/views/Hypergraph and multigraph analysis adapters
parser/One format-selecting facade over RDF, JSON, and text adapters; RDF schema declarations stay outside the concept graph
memory/Query cache: hot questions, short-term concept sets, and long-term patterns
routing/Harness of typed questions that replays a cached question or continues into retrieval
memory_store.pySaved concept embeddings, separate from the query cache
embeddings/encoders/Text and Node2Vec graph encoders
embeddings/vector_index/Backend-neutral vector search with an exact Torch implementation
retrieval/Structural sampling, web and vector retrieval, and an optional Jev context harness
functions/similarity/Cosine, normalized web distance, hybrid, and cross-graph scoring
prompting/Typed ontology-equivalence prompts and machine-readable answer parsing
validation/Alignment metrics, validation rules, tuple evaluation, pipelines, and ontology linting
uq/Dependency-free agreement scoring and optional UQLM, graph, and ensemble backends

Algorithm extension points

Graph domain

Extend focused models or use OntologyStructureBuilder to construct new graph representations.

Relation refinement

Configure RelationRefiner with a RuleSet and inspect its structured result.

Ontology refinement

Use OntologyRefiner to refine, rebuild, and preserve ontology metadata.

Offline graph refinement

Use OfflineGraphRefiner for deterministic structural collapse and partition refinement over a complete graph.

Online ontology refinement

OnlineOntologyRefiner follows KROMA Algorithm 2: recompute and sort ranks, split changed classes, apply rank-specific local merges, and defer low-confidence, cyclic, or rank-inconsistent updates. Both refiners accept configuration W's concept_similar predicate.

Vector retrieval

Embedding generation and vector search are separate. Encoders create text and graph vectors, fusion combines them, and a registered vector-index backend performs retrieval.

Encoders

TextEmbedding and GraphEmbedding implement small encoder contracts and can be replaced independently.

Fusion

fuse_embeddings validates dimensions and applies the configured graph/text weighting.

Exact search

TorchExactVectorIndex provides deterministic cosine retrieval for current experiments and tests.

Additional backends

register_vector_index supports optional ANN, DiskANN, and vector-database adapters. Select one through index_backend on graph retrieval.

Retrieval plugins

Use create_web_search with static, dataset, mediawiki, or google-cse. Each provider has visible metadata describing network access and required environment variables. Use register_web_search for a project-specific index. Normalized Google Distance receives the selected provider explicitly and does not hide a network service inside similarity code.

Memory and cache

The Agentic Ontologist keeps two stores for one ontology. Embedding memory holds the concept vectors so the next run can skip embedding. The query cache holds answers so a repeated question can skip the model. They live in different folders and are not interchangeable.

flowchart LR
    Ask["Ask"] --> Hot["Hot hash"]
    Hot -->|"same question"| Answer["Answer"]
    Hot -->|"miss"| Check["Shingle checksum"]
    Check -->|"names the concept"| Answer
    Check -->|"continue"| Harness["Harness"]
    Harness -->|"replay"| Answer
    Harness -->|"retrieve"| Retrieve["Retrieve concepts"]
    Retrieve --> Replay["Graph replay"]
    Replay -->|"same concepts"| Answer
    Replay -->|"new concepts"| Model["Model"]
    Model --> Short["Short-term trie"]
    Short -->|"seen 3 times"| Long["Long-term hash"]

    classDef boundary fill:#f6f8fa,stroke:#8c959f,color:#1f2328;
    classDef core fill:#ddf4ff,stroke:#0969da,color:#1f2328;
    class Ask,Answer,Model boundary;
    class Hot,Check,Harness,Retrieve,Replay,Short,Long core;
      

Replay reads a stored capture. It does not send the SPARQL pattern to an endpoint.

Embedding memory

results/memory stores one folder per ontology: manifest.json and embeddings.pt. agonto memory clear deletes that folder. --no-save-memory keeps the vectors in the current process only.

Hot

hot.json maps a normalized question to its answer. Matching ignores case and extra spaces. The cap is 32 entries. The default policy drops the least frequently read question.

Short-term

short_term.json is a trie of concept ids, the SPARQL pattern, and the answer. Lookup uses the set of concept ids, so order from retrieval does not matter. The cap is 512 nodes.

Long-term

long_term.json is a persistent hash of completed concept sets. A set is copied here after it has been answered 3 times. The cap is 1024 entries. A later question with those concepts replays from here after the short-term trie has dropped them.

One ontology is one directory, results/cache/<sha>/, with hot.json, short_term.json, and long_term.json. The first model answer writes the capture. The same question hits the hot tier before retrieval. After a miss, a checksum of three-word shingles on each remembered answer can name the last concept on that path and skip retrieval. The overlap must be at least 0.75 and lead the next answer by 0.15. A single misspelled word can still match. A question with fewer than 4 shingles continues. That checksum is rebuilt in memory from the concept trie and the long-term patterns. It is not a new cache file. The Harness then asks typed questions about the question the user typed and can replay a cached one. It runs only when the hot cache already has questions. A failed call, or a choice that is not in the cache, continues into retrieval. The default call is SGLang /v1/systemone at http://127.0.0.1:30000 (AGONTO_HARNESS_URL). An empty value keeps the Ollama router, ollama:phi3 (AGONTO_ROUTER_MODEL). Optional Jev routing still chooses among answer models inside the agent and stays separate from this step. A wording that is not replayed hits graph replay when retrieval returns the same concepts. A different concept set still calls the answer model.

Eviction

Each tier drops records when it passes its cap. lfu removes the least frequently read record, then the one touched longest ago. lru removes the record that was read or written longest ago. 2q removes a one-time record before a repeated one. Hot, short-term, and long-term use AGONTO_CACHE_EVICTION, which defaults to lfu. A new policy is a class with choose(entries) registered under a name. A new tier is a new file under agentic_ontologist.memory.cache.

AGONTO_CACHE_DIR moves the folder. AGONTO_HOT_ENTRIES, AGONTO_SHORT_TERM_NODES, and AGONTO_LONG_TERM_ENTRIES set the three caps.

Agent runtime

The public agentic_ontologist.agents API exposes the agent, observers, Ontology-of-Thought models, and supported tools. Internally, the agent emits each event once, the OOT observer owns conversation memory, and the thought graph is derived from that memory for persistence and query optimization. Optional Jev routing and gating add model.routed and tool.blocked to that same event stream. They stay off unless configured, and they do not replace the context harness that selects evidence before prompting.

flowchart TB
    User["User query"] --> Router["Optional model router"]
    Router --> Core["Agent"]
    Core --> Runtime["ModelRuntime"]
    Runtime --> LLM["Selected LLM"]

    Core --> Gate["Optional tool-risk gate"]
    Gate --> Executor["ToolExecutor"]
    Executor --> Registry["Tool registry"]
    Registry --> OntTools["Ontology providers"]
    Registry --> KGTools["Knowledge graph providers"]
    Registry --> WebTools["Website providers"]

    Core --> Events["Typed event stream"]
    Events --> Observer["OOTAgentObserver"]
    Observer --> Memory["MemoryTrace"]
    Memory --> Thoughts["ThoughtGraph"]
    Thoughts --> Optimizer["QueryOptimizer"]

    Core --> Persistence["Trace persistence"]
    Persistence --> Local["Local JSON"]
    Persistence --> Duck["Optional DuckDB"]
    Persistence --> Dynamo["Optional DynamoDB"]

    classDef external fill:#f6f8fa,stroke:#8c959f,color:#1f2328;
    classDef core fill:#ddf4ff,stroke:#0969da,color:#1f2328;
    class User,LLM,OntTools,KGTools,WebTools,Local,Duck,Dynamo external;
    class Core,Router,Runtime,Gate,Executor,Registry,Events,Observer,Memory,Thoughts,Optimizer,Persistence core;
      

Core

Agent coordinates the model and delegates registry-backed calls to the tool executor.

Routing

When enabled, Jev chooses local, standard, or careful once per run and records model.routed.

Gating

Configured external lookups can be refused before execution. The refusal is tool.blocked on the same trace.

Events

Observers receive one event stream without depending on the agent implementation.

Memory

MemoryTrace is the canonical record used to derive a ThoughtGraph.

Persistence

AGONTO_TRACE_STORE=duckdb writes each thought graph to results/traces/agentic_ontologist.duckdb. DynamoDB is used when a table is configured.

Agent tools

The Agentic Ontologist intentionally includes only tools used by ontology integration:

General SPRINT tools for files, geospatial data, PDFs, vision, tabular data, and unrelated workflows are not included.

Extension patterns