102 lines
6.3 KiB
Markdown
102 lines
6.3 KiB
Markdown
# BuddhaGPT — Design Spec
|
||
|
||
**Date:** 2026-08-14 · **Owner:** Marcus · **Linear:** PAI-87 (project BuddhaGPT)
|
||
|
||
## Goal
|
||
|
||
Show end-to-end LLM competency (prompting, fine-tuning, embeddings, retrieval, evaluation) plus product judgment, via a "Buddha GPT": an open 7B model fine-tuned on Buddhist literature, grounded by RAG over the Pali Canon, evaluated for compassion against base and frontier models, with a safety-research angle (does value-laden fine-tuning shift safety behavior?).
|
||
|
||
## Non-goals
|
||
|
||
- Training from scratch on Buddhist text only (corpus too small; proves nothing).
|
||
- Claiming the model "is" compassionate — we measure judged behavior on a defined rubric, and report sycophancy separately.
|
||
- Multi-tradition completeness. Scope: Theravada (Pali Canon) primary, clearly stated.
|
||
|
||
## Deliverables
|
||
|
||
1. **Public repo + writeup** — pipeline code, results, README (portfolio).
|
||
2. **Live demo** — Hugging Face Space (Gradio), merged model + RAG citations.
|
||
3. **Research-style report** — CompassionBench results + safety-benchmark deltas, HF model card.
|
||
|
||
## Constraints
|
||
|
||
- **Compute:** local Apple M5, 24 GB unified memory. Training via MLX (`mlx_lm.lora`), 4-bit QLoRA. No GPU rental.
|
||
- **Budget:** ~$100 ceiling, ~$6 expected — all API via OpenRouter: generation on `deepseek/deepseek-v4-flash-latest` (~$0.08/$0.16 per MTok), judge on `google/gemini-flash-latest`, frontier reference `moonshotai/kimi-k3`, second-judge agreement on `deepseek/deepseek-v4-pro`.
|
||
- **License hygiene:** corpus must be redistributable (CC0/CC-BY); base model Apache-2.0.
|
||
|
||
## Architecture
|
||
|
||
```
|
||
corpus (SuttaCentral/Bilara, Access to Insight, Dhammapada)
|
||
├─► data pipeline ─► instruction pairs (synthetic Q&A via DeepSeek V4 Flash, filtered)
|
||
│ └─► MLX QLoRA fine-tune of Qwen2.5-7B-Instruct (4-bit)
|
||
└─► chunker ─► embeddings (bge-small-en / mlx) ─► LanceDB index
|
||
│
|
||
user query ─► retrieval (top-k + citations) ─► fine-tuned model ─► answer + sutta refs
|
||
│
|
||
eval harness (CompassionBench + safety suite)
|
||
```
|
||
|
||
### Components
|
||
|
||
| Component | Choice | Why |
|
||
|---|---|---|
|
||
| Base model | Qwen2.5-7B-Instruct (mlx-community 4-bit) | Apache-2.0 (clean for public repo/demo), strong instruct base, fits 24 GB |
|
||
| Fine-tune | `mlx_lm.lora` QLoRA, ~5–10k pairs | Runs locally on M5; hours per run |
|
||
| Instruction data | DeepSeek V4 Flash via OpenRouter generating Q&A grounded in canon passages | ~$1-2 for ~9k pairs, quality controllable, filterable |
|
||
| Embeddings | `bge-small-en-v1.5` (or nomic-embed) local | Free, fast on M5 |
|
||
| Vector store | LanceDB | Embedded, no server, ships with the Space |
|
||
| Judge | `google/gemini-flash-latest` with rubric; `deepseek/deepseek-v4-pro` second-judge subset | Cheap, capable, independent of all compared systems; human-rated subset checks agreement |
|
||
| Demo | HF Space (Gradio) with merged 4-bit model | $0 hosting path (ZeroGPU); account `mrmen` exists |
|
||
|
||
### Fine-tune vs RAG split (a deliberate write-up point)
|
||
|
||
Fine-tuning carries voice, framing, and dharma-teacher persona. RAG carries facts and citations (real sutta references, e.g. "MN 21"). The eval compares FT-only, RAG-only, and FT+RAG to demonstrate the judgment of when each tool applies.
|
||
|
||
## Corpus
|
||
|
||
- **SuttaCentral / Bilara data** (GitHub `suttacentral/bilara-data`): Sujato translations of the four Nikāyas — CC0. Primary source.
|
||
- **Dhammapada** + selected Khuddaka texts (public-domain translations).
|
||
- **Access to Insight** (Thanissaro): free-distribution license — verify redistribution terms before inclusion; fallback is prompt-only use (not redistributed).
|
||
- Local seed: `~/buddhadasa_mindfulness-with-breathing.pdf` — check license; likely reference-only, not in training set.
|
||
|
||
## Evaluation design
|
||
|
||
### CompassionBench (custom, ~150 prompts, 5 categories)
|
||
|
||
1. Distressed user (grief, anxiety, loneliness)
|
||
2. Moral dilemma
|
||
3. Harmful request (refusal quality + tone)
|
||
4. Sycophancy trap ("tell me my bad plan is good") — measures compassion ≠ agreement
|
||
5. Existential/meaning questions
|
||
|
||
Systems compared: base Qwen2.5-7B, BuddhaGPT-FT, BuddhaGPT-FT+RAG, Kimi K3 (frontier reference). Judge: Gemini Flash with rubric scoring empathy, non-harm, honesty-under-pressure, groundedness — deliberately independent of every compared system (no self-preference bias). Agreement checks: Marcus rates a ~30-item subset (judge-human) and DeepSeek V4 Pro re-judges a 100-item subset (judge-judge).
|
||
|
||
### Safety delta (research angle)
|
||
|
||
Run the same safety probes (refusal set, sycophancy set, a TruthfulQA-style subset) on base vs fine-tuned. Question: does compassion-corpus fine-tuning measurably shift refusals, sycophancy, honesty? Either direction is a reportable finding (persona fine-tunes are known to sometimes degrade safety behavior).
|
||
|
||
## Product judgment / guardrails
|
||
|
||
- Citations mandatory in RAG mode; answers without a retrieved source are labeled as such.
|
||
- Demo disclaimer: not a teacher, not therapy; crisis-resources footer for distress-adjacent inputs.
|
||
- Scope statement: Theravada corpus; answers reflect that tradition.
|
||
|
||
## Error handling / risks
|
||
|
||
| Risk | Mitigation |
|
||
|---|---|
|
||
| M5 training too slow / OOM | 4-bit base + LoRA rank ≤ 16, batch 1 + grad accumulation; shrink dataset before shrinking model |
|
||
| Synthetic data mode-collapse (samey Q&A) | Diverse prompt templates, dedupe by embedding similarity, temperature-free variety via varied instructions |
|
||
| Judge bias toward flowery tone | Rubric penalizes vagueness; judge independent of all compared systems; human + second-judge agreement subsets |
|
||
| ZeroGPU Space limits (7B latency/quota) | Fallback: demo on 3B (Qwen2.5-3B) for the Space, 7B results in the report; or recorded demo |
|
||
| Eval bank contamination (prompts leak style) | Hold eval prompts out of all training data; build them after data-gen prompts frozen |
|
||
|
||
## Milestones
|
||
|
||
- **M1 — Corpus + RAG MVP:** indexed canon, cited retrieval answers over base model.
|
||
- **M2 — Fine-tune v1:** 5k pairs, QLoRA run, qualitative diff vs base.
|
||
- **M3 — Eval:** CompassionBench + safety deltas, judge + human subset.
|
||
- **M4 — Demo:** HF Space with FT+RAG, guardrails, disclaimer.
|
||
- **M5 — Writeup:** README, report, model card, publish.
|