Skip to content

CSED Knowledge Base

The CSED Knowledge Base is a federated cognitive system that serves as the lab's living memory — a structured, versioned repository of architectural decisions, research context, design rationale, and operational knowledge.

Architecture

The system is built on three core components:

  • OKF (Open Knowledge Format) — A Markdown + YAML frontmatter standard for representing knowledge nodes, validated against the Google Open Knowledge Format specification (released June 2026). Each node carries a dual-label type array for faceted classification.
  • kb.toml Manifests — Dependency manifests using registry:// URIs that declare static and dynamic knowledge dependencies, resolved via $KB_ROOT and managed with the kb CLI toolchain (Knowledge Base Manager / KBM).
  • Hybrid GraphRAG — A dual-store retrieval architecture combining Qdrant (vector embeddings for semantic similarity search) with Neo4j (graph traversals for relationship-aware queries). This enables both "find documents similar to this one" and "trace the chain of decisions that led to X" queries.

Design Principles

The KB architecture is grounded in the Interpretable Context Methodology (ICM) (arXiv:2603.16021v2), which formalizes the principle that folder structure is agent architecture.

The Input/Output Firewall: A strict separation between immutable ground-truth assets (sources/ or db/) and mutable AI/human iterations (work/ or workspace/) is mandatory. This prevents ingestion pipelines from feeding unverified AI-generated drafts back into the knowledge graph, avoiding recursive hallucination loops. Files in sources/ are indexed as authoritative nodes; files in workspace/ are tagged as derivative work with DERIVED_FROM edges pointing back to their sources.

The 3-Tier Knowledge OS: Knowledge is organized across three tiers — Project Level (active, 1-6 months), Domain Level (permanent discipline knowledge), and Master Research Vault (global synthesis). This prevents monolithic knowledge bases from becoming unwieldy and allows agents to scope their context appropriately.

Organizational Memory: AI chat logs and web clippings are treated as first-class citizens alongside formal papers. The KB schema includes AIChatLog and WebResource node types specifically to capture infrastructure commands, design debates, and cluster quirks that never make it into publications.

Entity-Relationship Schema

The knowledge graph defines seven primary node types — ResearchPaper, Concept, Project, Methodology, Dataset, AIChatLog, and WebResource — connected by standard predicates: addressesProblem, usesMethod, evaluatesOn, implements, supportsClaim, and cites. These directional verbs turn the isolated database into a traversable semantic graph, enabling queries like "find all concepts evaluated by methodology X in research paper Y."

Key Design Decisions

Decision Choice
Registry ArangoDB (KnowledgeNodes / KnowledgeEdges) — no flat kb.json file
Addressing registry:// URIs resolved via $KB_ROOT
Edge syntax [predicate::Display Title](href) — href authoritative, bare links default to discusses
Sync semantics kb sync respects kb.lock; kb update re-resolves dynamic deps
Versioning Append-only nodes (version / supersedes / is_latest) ≅ prov:wasRevisionOf
Vault storage Box (interim) → Ceph S3 (target)

Implementation

The Knowledge Base is implemented through two repositories:

  • csed-ucm/kb — The pilot repository containing architectural specs, OKF schemas, dual-label ontology definitions, glossary, and strategy documents. This is the canonical reference for the KB architecture.
  • csed-ucm/kbm — The Knowledge Base Manager (KBM) — a Python CLI tool (kb sync, kb update, kb init) that serves as the orchestration engine. It parses kb.toml manifests and OKF Markdown files, extracts typed nodes and directional edges, and manages synchronization with the graph registry (ArangoDB). Currently in active development.

Document Parsing Pipeline

The KB ingestion pipeline is informed by ongoing research into state-of-the-art document parsing. See the Document Parsing SOTA report for a survey of DeepSeek-OCR 2, Marker, DELM, and other 2026 frameworks.

Supplementary Reading