kroma.inference

Models

Chat providers adapt OpenAI and Together. Embedding providers adapt Hugging Face and sentence-transformers.

Factories

create_inference_model(backend, model_name) -> BaseModel
create_embedding_model(backend, model_name) -> BaseEmbeddingModel
register_chat_backend(name, provider_cls) -> None
register_embedding_backend(name, provider_cls) -> None

Chat backends are openai and togetherai. The API key is OPENAI_API_KEY or TOGETHERAI_API_KEY. Embedding backends are huggingface and sentence-transformers.

BaseModel.generate retries transient connection, timeout, and rate-limit errors.

Source: factory.py.

Inference

infer(debate, backend, model_name, source_term, target_term, ...)
  -> prediction, metrics, confidence, accept

Class	Role
`SingleModelInference`	One agent, one round.
`DebateInference`	One agent per config, for `n_rounds` rounds.

expand_term calls query and returns the concept code plus parents, children, synonyms, and labels.

Source: inference.py.