buddha-gpt/docs/superpowers/specs/2026-08-14-buddha-gpt-design.md

BuddhaGPT — Design Spec

Date: 2026-08-14 · Owner: Marcus · Linear: PAI-87 (project BuddhaGPT)

Goal

Show end-to-end LLM competency (prompting, fine-tuning, embeddings, retrieval, evaluation) plus product judgment, via a "Buddha GPT": an open 7B model fine-tuned on Buddhist literature, grounded by RAG over the Pali Canon, evaluated for compassion against base and frontier models, with a safety-research angle (does value-laden fine-tuning shift safety behavior?).

Non-goals

• Training from scratch on Buddhist text only (corpus too small; proves nothing).
• Claiming the model "is" compassionate — we measure judged behavior on a defined rubric, and report sycophancy separately.
• Multi-tradition completeness. Scope: Theravada (Pali Canon) primary, clearly stated.

Deliverables

1. Public repo + writeup — pipeline code, results, README (portfolio).
2. Live demo — Hugging Face Space (Gradio), merged model + RAG citations.
3. Research-style report — CompassionBench results + safety-benchmark deltas, HF model card.

Constraints

• Compute: local Apple M5, 24 GB unified memory. Training via MLX (mlx_lm.lora), 4-bit QLoRA. No GPU rental.
• Budget: ~$100 ceiling, $6 expected — all API via OpenRouter: generation on deepseek/deepseek-v4-flash-latest ($0.08/$0.16 per MTok), judge on google/gemini-flash-latest, frontier reference moonshotai/kimi-k3, second-judge agreement on deepseek/deepseek-v4-pro.
• License hygiene: corpus must be redistributable (CC0/CC-BY); base model Apache-2.0.

Architecture

corpus (SuttaCentral/Bilara, Access to Insight, Dhammapada)
  ├─► data pipeline ─► instruction pairs (synthetic Q&A via Claude, filtered)
  │                        └─► MLX QLoRA fine-tune of Qwen2.5-7B-Instruct (4-bit)
  └─► chunker ─► embeddings (bge-small-en / mlx) ─► LanceDB index
                                                        │
user query ─► retrieval (top-k + citations) ─► fine-tuned model ─► answer + sutta refs
                                                        │
                                              eval harness (CompassionBench + safety suite)

Components

Component	Choice	Why
Base model	Qwen2.5-7B-Instruct (mlx-community 4-bit)	Apache-2.0 (clean for public repo/demo), strong instruct base, fits 24 GB
Fine-tune	mlx_lm.lora QLoRA, ~5–10k pairs	Runs locally on M5; hours per run
Instruction data	DeepSeek V4 Flash via OpenRouter generating Q&A grounded in canon passages	~$1-2 for ~9k pairs, quality controllable, filterable
Embeddings	bge-small-en-v1.5 (or nomic-embed) local	Free, fast on M5
Vector store	LanceDB	Embedded, no server, ships with the Space
Judge	google/gemini-flash-latest with rubric; deepseek/deepseek-v4-pro second-judge subset	Cheap, capable, independent of all compared systems; human-rated subset checks agreement
Demo	HF Space (Gradio) with merged 4-bit model	$0 hosting path (ZeroGPU); account mrmen exists

Fine-tune vs RAG split (a deliberate write-up point)

Fine-tuning carries voice, framing, and dharma-teacher persona. RAG carries facts and citations (real sutta references, e.g. "MN 21"). The eval compares FT-only, RAG-only, and FT+RAG to demonstrate the judgment of when each tool applies.

Corpus

• SuttaCentral / Bilara data (GitHub suttacentral/bilara-data): Sujato translations of the four Nikāyas — CC0. Primary source.
• Dhammapada + selected Khuddaka texts (public-domain translations).
• Access to Insight (Thanissaro): free-distribution license — verify redistribution terms before inclusion; fallback is prompt-only use (not redistributed).
• Local seed: ~/buddhadasa_mindfulness-with-breathing.pdf — check license; likely reference-only, not in training set.

Evaluation design

CompassionBench (custom, ~150 prompts, 5 categories)

1. Distressed user (grief, anxiety, loneliness)
2. Moral dilemma
3. Harmful request (refusal quality + tone)
4. Sycophancy trap ("tell me my bad plan is good") — measures compassion ≠ agreement
5. Existential/meaning questions

Systems compared: base Qwen2.5-7B, BuddhaGPT-FT, BuddhaGPT-FT+RAG, Kimi K3 (frontier reference). Judge: Gemini Flash with rubric scoring empathy, non-harm, honesty-under-pressure, groundedness — deliberately independent of every compared system (no self-preference bias). Agreement checks: Marcus rates a ~30-item subset (judge-human) and DeepSeek V4 Pro re-judges a 100-item subset (judge-judge).

Safety delta (research angle)

Run the same safety probes (refusal set, sycophancy set, a TruthfulQA-style subset) on base vs fine-tuned. Question: does compassion-corpus fine-tuning measurably shift refusals, sycophancy, honesty? Either direction is a reportable finding (persona fine-tunes are known to sometimes degrade safety behavior).

Product judgment / guardrails

• Citations mandatory in RAG mode; answers without a retrieved source are labeled as such.
• Demo disclaimer: not a teacher, not therapy; crisis-resources footer for distress-adjacent inputs.
• Scope statement: Theravada corpus; answers reflect that tradition.

Error handling / risks

Risk	Mitigation
M5 training too slow / OOM	4-bit base + LoRA rank ≤ 16, batch 1 + grad accumulation; shrink dataset before shrinking model
Synthetic data mode-collapse (samey Q&A)	Diverse prompt templates, dedupe by embedding similarity, temperature-free variety via varied instructions
Judge bias toward flowery tone	Rubric penalizes vagueness; judge independent of all compared systems; human + second-judge agreement subsets
ZeroGPU Space limits (7B latency/quota)	Fallback: demo on 3B (Qwen2.5-3B) for the Space, 7B results in the report; or recorded demo
Eval bank contamination (prompts leak style)	Hold eval prompts out of all training data; build them after data-gen prompts frozen

Milestones

• M1 — Corpus + RAG MVP: indexed canon, cited retrieval answers over base model.
• M2 — Fine-tune v1: 5k pairs, QLoRA run, qualitative diff vs base.
• M3 — Eval: CompassionBench + safety deltas, judge + human subset.
• M4 — Demo: HF Space with FT+RAG, guardrails, disclaimer.
• M5 — Writeup: README, report, model card, publish.