Knowledge — AI API Guide

Written for an AI agent. Read top to bottom once; then use §3 (playbook) and §5 (reference). Companion to apps/api/guides/simple-flow/AI_API_GUIDE_WORLFLOWS.md (flows) — same key model, same conventions.

A knowledge base is a set of Q&A entries (subject · question · answer, vector + keyword indexed) that Simple Flow LLM nodes and the “Ask” endpoint retrieve from. Entries are created by submitting material (text, or an uploaded document’s text) — the AI reads it and extracts pairs — or written directly. This API lets you list and create knowledge bases and read, search, ask, add, edit, delete entries, submit new material, import/export, and see the questions a base could not answer — using a personal API token the user created on Settings → API.

Vocabulary you will meet: flow = a Simple Flow automation (chatbot) that calls this base at runtime; LLM node = the step in a flow that decides to search; studio = the user’s web UI for this base (chat-style, with a “Test” drawer for trying questions). None of these are needed to use the API.

---

0. Quick start for agents

1. GET  /me                                   → confirm the token works; note `access` (read | write)
2. GET  /bases                                → every base in the org; pick the kbId (or POST /bases to create one — write)
   GET  /bases/{kbId}                         → name, description, entryCount, reviewBeforeIndex
3. GET  /bases/{kbId}/subjects                → the taxonomy in use (reuse subjects, don't invent near-duplicates)
4. Retrieve:  POST /bases/{kbId}/search { query }   → ranked entries (no credits)
              POST /bases/{kbId}/ask    { query }   → grounded answer + the entries behind it (credits)
5. Grow — pick by what you have:
     ready Q&A pairs, one   → POST /bases/{kbId}/entries { subject, question, answer }   (indexed immediately, embedding only)
     ready Q&A pairs, many  → POST /bases/{kbId}/import  { rows: [{ subject?, question, answer }] }   (≤ 2000 per call, embedding only)
     raw material / a doc   → POST /bases/{kbId}/submissions { text }   (AI extracts the pairs — LLM credits)
6. Maintain:  GET/PATCH/DELETE entries, POST entries/bulk, GET unanswered (the gaps)
7. Report what you changed by entry id + question text

Do not send ready-made Q&A through /submissions — that runs LLM extraction over your text, costs credits, and may reword or split your pairs. /entries and /import store them verbatim.

Retrieval is hybrid: cosine similarity on embeddings + Postgres full-text, fused with RRF. vectorScore is the cosine similarity (0–1) when the vector leg matched.

---

1. Credentials

The user creates a personal API token in the app: avatar menu → API → Create token (name · access Read only / Read & write · resources Knowledge / Workflows · expiry). The page’s Copy JSON gives:

{
  "apiToken": "djc_…",
  "orgId": "d3fe674f-…",
  "userId": "Qrpc3k06…",
  "resources": {
    "knowledge": { "access": "write", "baseUrl": "https://<domain>/api/ai/knowledge", "guide": "apps/api/guides/knowledge/AI_API_GUIDE_KNOWLEDGE.md", "firstCall": "GET /me" }
  }
}

A knowledge base’s own ⚙ Settings → AI access tab gives the same shape plus knowledgeBaseId for that base.

Value	Use
apiToken	Every request: Authorization: Bearer djc_… (or header X-API-Key: djc_…). Local dev base URL: http://localhost:8787/api/ai/knowledge.
access	read → GET routes plus POST /search and POST /ask. write → everything (create bases, entries, submissions, import, bulk, delete). A write call with a read token → 403 INSUFFICIENT_SCOPE.
orgId, userId	Informational. The token acts as this user in this org — it sees every base in the org; extraction, embeddings and Ask spend this user’s credits in this org (402 when the balance is ≤ 0).

Sanity check:

BASE=…; H='Authorization: Bearer djc_…'
curl -s -H "$H" $BASE/me
# {"userId":"…","orgId":"…","tokenKind":"token","access":"write","scopes":["knowledge:write"],"knowledgeBaseId":null,"base":"/bases"}
curl -s -H "$H" $BASE/bases        # [{ id, name, description, entryCount, … }]

Legacy keys (kbk_…) still work but are bound to one base: /me returns that knowledgeBaseId, GET /bases and POST /bases answer 403, and access is always write for that base. If the token is placeholder text (<create or reveal a token …>), ask the user to create one on Settings → API.

---

2. Data model — read this before editing

Base

{ id, orgId, name, description, embeddingModel, reviewBeforeIndex, defaultLanguage, sourceCount, entryCount, chunkCount, indexing, lastIndexedAt, createdAt, updatedAt } (chunkCount mirrors entryCount; ignore it)

• description is shown to LLM nodes so they know when to search this base — keep it accurate when you change what the base covers.
• reviewBeforeIndex: true → extracted pairs land as drafts; flows only see active entries. Approve with POST entries/bulk { action: "approve" }.
• defaultLanguage → extraction writes questions/answers in this language (else the source language).

Entry (the indexed unit)

{
  "id": "uuid", "subject": "Pricing", "question": "What is the warranty on the X200?",
  "altQuestions": ["X200 warranty length", "How long is the X200 covered?"],
  "answer": "Two years from the date of purchase, parts and labour.",
  "keywords": ["X200", "warranty"],
  "sourceExcerpt": "…the X200 carries a two-year warranty…", "page": 4,
  "status": "active", "conflictOfId": null, "validUntil": null,
  "hitCount": 3, "lastHitAt": "2026-08-29T07:30:00Z",
  "sourceId": "uuid|null", "sourceTitle": "X-series datasheet", "messageId": "uuid|null",
  "createdAt": "…", "updatedAt": "…"
}

Field	Rules
subject	1–60 chars, Title Case, one or two words (“Pricing”, “Warranty”, “Office Hours”). Reuse what GET /subjects returns. It is part of the embedding text, so changing it re-embeds.
question	1–500 chars, one clear question a customer would actually ask. Exact duplicates (case-insensitive, vs non-archived entries) are rejected — 400 on POST /entries, counted in duplicates on /import and /submissions.
altQuestions	≤ 5 paraphrases, ≤ 300 chars each — improve recall for different wordings/languages.
answer	1–2000 chars, complete and self-contained (the LLM sees only matched entries, never the source).
keywords	≤ 10, ≤ 60 chars — product codes, names, numbers that keyword search should hit. On /import an over-long list fails the whole batch with a message that blames rows (§4g).
status	active (searchable) · draft (hidden until approved) · archived (retired; excluded from search and counts).
conflictOfId	Set when a new entry was ≥ 0.92 cosine-similar to an existing one — it is stored as a draft “possible duplicate”. Resolve with `PATCH { resolveConflict: “keep”
validUntil	ISO datetime or null — promotions, seasonal facts. Listed under status=expiring when < 30 days away.
hitCount	Times retrieval returned it. sort=unused finds dead weight.

POST/PATCH responses additionally carry knowledgeBaseId, orgId, createdByUserId, updatedByUserId — bookkeeping, safe to ignore. Embedding vectors are never returned.

Sources, messages

• A source is one piece of submitted material: kind: "file" (uploaded document, original kept in storage) or "text" (pasted text). Entries point back to it (sourceId, page, sourceExcerpt).
• A message is a studio chat turn. Submissions create a user turn (the material) and an assistant turn (the extraction report). Deleting a message keeps its entries.

Search hit

{ id, subject, question, answer, altQuestions, sourceId, sourceTitle, page, vectorScore: number|null, score } — score is the fused rank (ordering only). Every retrieval bumps hitCount; when nothing clears minScore the query is logged under unanswered (gaps).

---

3. Playbook — how to work on a user’s base

1. Inspect first: GET /bases to find the base (create one with POST /bases { name, description } only if the user asked for a new one), then GET /bases/{kbId}, GET /subjects and GET /entries?limit=50. Summarise size, subjects and obvious gaps before changing anything the user didn’t ask about.
2. Search before you add. POST /search { query } with the question you’re about to write. If a near-identical entry exists, PATCH it instead of creating a competing one — two answers to one question makes retrieval worse, not better.
3. Bulk material → /submissions, not hundreds of POST /entries. Send clean text (headings and paragraphs; strip navigation/boilerplate). Up to 400 000 characters per submission; ~6 000-token windows are extracted in parallel. Prefer 5–40 k characters per call for the best pair quality. Add an instruction in text only when uploading files (see §4b); with plain text the text is the material.
4. Curate after extraction: read the returned entries, fix wording with PATCH, merge near-duplicates, set subjects consistently (POST entries/bulk { action: "setSubject" }). If the base uses review mode, approve the good drafts (bulk approve) and delete the rest.
5. Ready Q&A goes in verbatim: one fact → POST /entries; a list → POST /import (see §4g). One fact per entry; put the variants in altQuestions, not in separate entries. Both paths run the same duplicate check as extraction (§2 conflictOfId), so read the conflicts count and resolve GET /entries?status=conflicts afterwards.
6. Close the gaps: GET /unanswered lists real questions retrieval failed on (from flows, the Test drawer and this API). Write entries for them, then DELETE /unanswered?query=… (or all).
7. Test like a flow does: POST /ask { query } returns exactly what a flow’s Knowledge tool would ground on. POST /search is free — use it for iteration; ask and submissions cost credits.
8. Costs: extraction ≈ input tokens of the material + ~⅓ output, plus embeddings (tiny). A 402 means the user’s balance in this org is ≤ 0 — stop and tell them; don’t retry.
9. Concurrency: the user’s studio shows your entries within seconds (it reloads on focus/actions). Writes are last-wins; GET the entry before a PATCH if the user may be editing it.
10. Don’t: invent subjects that already exist under another spelling, paste whole documents into one answer, archive/delete entries you haven’t read, or change description to something the user didn’t ask for. A base cannot be deleted through this API (DELETE /bases/{kbId} → 403).

---

4. Worked examples

4a. Seed a base from a policy text

BASE=…; H='Authorization: Bearer kbk_…'; KB=<knowledgeBaseId>
curl -s -H "$H" -H 'Content-Type: application/json' -X POST $BASE/bases/$KB/submissions -d '{
  "text": "## Returns policy\nItems can be returned within 30 days of delivery in original packaging. Refunds are issued to the original payment method within 5 working days of receipt. Sale items are exchange-only.\n\n## Shipping\nWest Malaysia: RM 8 flat, 2–4 working days. East Malaysia: RM 15, 4–7 working days. Free shipping above RM 150."
}'

Response (one JSON document — the stream is folded for you):

{
  "status": "success",
  "userMessageId": "…", "assistantMessageId": "…",
  "title": "Returns & Shipping Policy",
  "summary": "Return window, refund timing, sale-item rule and domestic shipping rates.",
  "entries": [
    { "id": "…", "subject": "Returns", "question": "How long do I have to return an item?", "answer": "30 days from delivery, in original packaging.", "status": "active", … },
    { "id": "…", "subject": "Shipping", "question": "How much is shipping to East Malaysia?", "answer": "RM 15, 4–7 working days; free above RM 150.", … }
  ],
  "extraction": { "status": "done", "model": "…", "windows": 1, "pairsExtracted": 7, "pairsInserted": 7, "duplicates": 0, "conflicts": 0,
                  "inputTokens": 412, "outputTokens": 590, "embeddingTokens": 380, "credits": 0.02 },
  "statusLog": ["Extracting 1 of 1…", "Indexing 7 entries…"]
}

status: "error" carries error (a credits/LLM/quota message). Pass ?stream=1 to get the raw NDJSON event stream instead (message, status, entry, title, done, error lines).

4b. Upload a document, then extract with an instruction

The server does not parse PDFs — send the extracted text with the file (the studio does this in the browser). Plain .txt/.md/.csv/.json can be sent as-is.

# 1. upload: file + its text (+ pageCount for PDFs). Mark pages in the text with lines "--- page N ---" so entries get page numbers.
curl -s -H "$H" -X POST $BASE/bases/$KB/sources/upload \
  -F "[email protected]" -F "content=<datasheet.txt" -F "pageCount=12"
# → 201 { "id": "<sourceId>", "kind": "file", "title": "datasheet", "charCount": 48210, "status": "ready", … }

# 2. extract from it; `text` is now an INSTRUCTION, not material
curl -s -H "$H" -H 'Content-Type: application/json' -X POST $BASE/bases/$KB/submissions -d '{
  "text": "Focus on specifications, prices and warranty. Skip marketing copy.",
  "attachments": [{ "sourceId": "<sourceId>" }]
}'

Up to 8 attachments per submission. A source can be attached to one submission; re-upload to extract again.

4c. Fix a wrong answer and add a paraphrase

curl -s -H "$H" -H 'Content-Type: application/json' -X POST $BASE/bases/$KB/search -d '{"query":"warranty X200","topK":3}'
# pick the hit → id
curl -s -H "$H" -H 'Content-Type: application/json' -X PATCH $BASE/bases/$KB/entries/<id> -d '{
  "answer": "Two years from the date of purchase (parts and labour). Batteries: one year.",
  "altQuestions": ["X200 warranty length", "Is the X200 battery covered?"]
}'

Changing subject, question, answer or altQuestions re-embeds the entry automatically.

4d. Resolve duplicates flagged by extraction

curl -s -H "$H" "$BASE/bases/$KB/entries?status=conflicts"
# each item has conflictOfId → the older entry it resembles
curl -s -H "$H" -H 'Content-Type: application/json' -X PATCH $BASE/bases/$KB/entries/<newId> -d '{"resolveConflict":"replace"}'   # archives the old one, activates this
curl -s -H "$H" -H 'Content-Type: application/json' -X PATCH $BASE/bases/$KB/entries/<newId> -d '{"resolveConflict":"keep"}'      # both stay, flag cleared

4e. Answer the gaps

curl -s -H "$H" $BASE/bases/$KB/unanswered
# [{"query":"do you ship to singapore","count":4,"lastAt":"…"}]
curl -s -H "$H" -H 'Content-Type: application/json' -X POST $BASE/bases/$KB/entries -d '{
  "subject":"Shipping","question":"Do you ship to Singapore?",
  "altQuestions":["International shipping","Can I order from Singapore?"],
  "answer":"Not yet — we ship within Malaysia only. Join the newsletter to hear when international shipping opens.",
  "keywords":["Singapore","international"]
}'
curl -s -H "$H" -X DELETE "$BASE/bases/$KB/unanswered?query=do%20you%20ship%20to%20singapore"

4f. Ask — what a flow sees

curl -s -H "$H" -H 'Content-Type: application/json' -X POST $BASE/bases/$KB/ask -d '{"query":"How long is the X200 warranty?"}'
# {"answer":"Two years from purchase (parts and labour); batteries one year.","hits":[…],"usage":{…}}
# no match → {"answer":null,"hits":[],"note":"No entries matched this question."}  (and the query is logged under unanswered)

4g. Import a list of ready Q&A pairs

curl -s -H "$H" -H 'Content-Type: application/json' -X POST $BASE/bases/$KB/import -d '{
  "rows": [
    { "subject": "Shipping", "question": "Do you ship to Singapore?", "answer": "Not yet — Malaysia only.",
      "altQuestions": ["International shipping"], "keywords": ["Singapore"] },
    { "question": "What are your office hours?", "answer": "Mon–Fri 9am–6pm." }
  ]
}'
# → { "inserted": 2, "duplicates": 0, "conflicts": 0 }

What happens to each row:

• Exact-duplicate question (case-insensitive; already in the base — archived entries don’t count — or repeated in rows) → skipped, counted in duplicates.
• ≥ 0.92 cosine-similar to an existing entry → stored as a draft with conflictOfId, counted in conflicts; it is not searchable until you resolve it (§4d) or bulk approve it.
• Everything else → active immediately, even when the base has reviewBeforeIndex: true.
• Missing subject → "General". validUntil is not accepted per row — PATCH the entry afterwards.
• Per-row limits are validated for the whole batch: keywords ≤ 10 and altQuestions ≤ 5 per row (§2). One row over the limit rejects the entire call with 400 Invalid body — and the zod message is misleading: "rows": ["Array must contain at most 10 element(s)"] means a row has more than 10 keywords, not that rows is too long. Trim the offending row and resend; nothing from that call was inserted.
• Cost: embeddings only (no LLM). Refused with 400 once the base holds 20 000 entries.

---

5. Reference

All bodies/responses are JSON unless stated. Errors:

{ "success": false, "error": "human-readable message", "code": "BAD_REQUEST", "details": { …zod field errors… } }

400 invalid body / rule violation · 401 missing, revoked or expired token (no code field) · 402 INSUFFICIENT_CREDITS (balance in details.balance) · 403 INSUFFICIENT_SCOPE (read token on a write route — details.required names the scope), FORBIDDEN (legacy key on another base, base delete, feature not enabled for the org) · 404 base/entry/source not found.

Discovery

Method	Path	Returns
GET	/me	{ userId, orgId, tokenKind: "token"|"legacy", access: "read"|"write", scopes, knowledgeBaseId: string|null, base } — knowledgeBaseId is set only for legacy keys

Bases

Method	Path	Body → Returns
GET	/bases	Base[] — every base in the org, most recently updated first
POST	/bases	{ name (≤120), description? (≤2000), reviewBeforeIndex?, defaultLanguage? } → 201 Base. Write. Max 50 bases per org.
DELETE	/bases/{kbId}	always 403 — delete from the app
GET	/bases/{kbId}	Base (§2)
PATCH	/bases/{kbId}	{ name?, description?, reviewBeforeIndex?, defaultLanguage? } → Base

Entries

Method	Path	Body / query → Returns
GET	/bases/{kbId}/entries	?q= (question/answer/subject contains) &subject= &source=<sourceId> &status=active|draft|archived|all|conflicts|expiring (default: not archived) &sort=newest|hits|unused|subject &limit=1..500 &offset= → { items: Entry[], total }
GET	/bases/{kbId}/subjects	[{ subject, count }] (non-archived), most used first
POST	/bases/{kbId}/entries	{ subject, question, answer, altQuestions?, keywords?, validUntil? } → 201 Entry + conflicts (number). Always active. Exact-duplicate question → 400. Credits (embedding).
PATCH	/bases/{kbId}/entries/{id}	any of { subject, question, answer, altQuestions, keywords, validUntil, status, resolveConflict: "keep"|"replace" } → Entry. Text changes re-embed.
DELETE	/bases/{kbId}/entries/{id}	→ { ok: true }
POST	/bases/{kbId}/entries/bulk	{ ids: [≤500], action: "approve"|"activate"|"archive"|"delete"|"setSubject", subject? } → { affected }. approve = draft → active and clears conflict flags; setSubject re-embeds in the background.

Retrieval

Method	Path	Body → Returns
POST	/bases/{kbId}/search	{ query (≤1000), topK? (1–20, default 8), minScore? (0–1, default 0.35) } → { hits: Hit[], embeddingTokens }. No LLM call; embedding only (not charged). Logs misses under unanswered.
POST	/bases/{kbId}/ask	same body (default topK 6) → { answer: string|null, hits, usage?, note? }. The model answers only from the hits (“I don’t have that information.” when they don’t cover it). Credits.

Submissions (AI extraction)

Method	Path	Body → Returns
POST	/bases/{kbId}/submissions	{ text (≤400k chars), attachments?: [{ sourceId }] (≤8), webSearch?: boolean } → collected JSON (§4a). text is the material when there are no attachments, an instruction when there are. For ready-made Q&A use /entries or /import instead. webSearch: true lets the model add verified facts from the web (extra per-search fee; sourceExcerpt becomes "Web: <domain>"). Append ?stream=1 for NDJSON events. Credits; refuses at 20 000 entries per base.
GET	/bases/{kbId}/messages	?limit=1..200 (default 60) &before=<ISO datetime> (older than) → { items: [{ id, role: "user"|"assistant", content, attachments, extraction, entries: Entry[], createdAt }], hasMore } — oldest first; entries is filled on user turns only
DELETE	/bases/{kbId}/messages/{id}	removes the turn (and its paired reply); entries stay → { ok: true }

Sources

Method	Path	Body → Returns
GET	/bases/{kbId}/sources	[{ id, kind: "file"|"text", title, fileName, mimeType, sizeBytes, pageCount, charCount, entryCount, status, messageId, createdAt }] (never the text)
POST	/bases/{kbId}/sources/upload	multipart file (≤20 MB; pdf/doc/docx/xls/xlsx/csv/txt/md/json) + content (the extracted text, ≤400k chars, required) + pageCount? → 201 Source
GET	/bases/{kbId}/sources/{id}/download	the original file (binary)
DELETE	/bases/{kbId}/sources/{id}	deletes the source and its entries → { ok: true }

Import / export / gaps

Method	Path	Body → Returns
GET	/bases/{kbId}/export?format=csv|json	all non-archived entries (subject, question, answer, altQuestions, keywords, status, validUntil, hitCount, createdAt)
POST	/bases/{kbId}/import	{ rows: [{ subject?, question, answer, altQuestions?, keywords? }] (1–2000; per row altQuestions≤ 5,keywords ≤ 10 — one bad row fails the batch, see §4g) } → { inserted, duplicates, conflicts }. Rows land active (near-duplicates → draft conflicts, see §4g); missing subject → “General”; no validUntil per row. Credits (embeddings only).
GET	/bases/{kbId}/unanswered	[{ query, count, lastAt }] — up to 200, most frequent first
DELETE	/bases/{kbId}/unanswered?query=	clears one query (omit query to clear all) → { ok: true }

Limits (so you don’t look for them)

• No base deletion and no cross-base search through this API (Simple Flow’s Knowledge tool can read several bases) — search one base at a time.
• Extraction language: defaultLanguage of the base, else the source’s. Model choice is fixed server-side.
• Embeddings are 1024-d; vectorScore ≥ ~0.6 is a strong match, 0.35–0.6 is related, below is noise (and is dropped unless full-text also matched).
• The server never parses PDF/Word/Excel — upload the file with its text.

---

6. Changelog

• 2026-08-29 — /import: documented that per-row keywords / altQuestions limits reject the whole batch, and that the error text (rows: at most 10 element(s)) refers to a row’s keywords, not the row count (§2, §4g, §5).
• 2026-08-29 (c) — Personal API tokens (djc_…, Settings → API) with knowledge:read / knowledge:write scopes replace per-base keys; tokens see the whole org: GET /bases, POST /bases; /me reports access and scopes; 403 INSUFFICIENT_SCOPE. Legacy kbk_ keys still accepted (bound to one base).
• 2026-08-29 (b) — Ready-Q&A decision guide (§0 Quick start, §3, §4g); documented /messages paging shape ({ items, hasMore }); import duplicate/conflict semantics; glossary; chunkCount and bookkeeping fields noted; API responses no longer include embedding vectors.
• 2026-08-29 — Initial version: per-base API keys (kbk_…, Settings → AI access), /api/ai/knowledge with /me and every studio route under /bases/{kbId} (entries, subjects, search, ask, submissions with collected-JSON response, sources, messages, import/export, unanswered).