Answering newcomers from a guidebook, not from vibes
/ 2 min read
Table of Contents
Every year, a new batch of Indonesian students lands in Nanjing and asks the same twenty questions: which dorm, which mosque, how to get a SIM card, where to open a bank account. The answers already exist — in a guidebook nobody reads until they need it.
So the portal answers them, but only from the guidebook.
Retrieval before generation
A general-purpose model will happily invent a visa rule. So the chat pipeline is:
- Chunk the guidebook into retrievable passages, keeping the heading context attached.
- Search the corpus for the passages that match the question.
- Generate an answer grounded in those passages, through a model fallback chain so a retired model does not take the feature down.
If nothing relevant is retrieved, the right answer is “not in the guidebook” — not a confident guess.
Guardrails that matter more than the model
- A daily question limit per session, because an unbounded chat is an unbounded bill.
- Answers cite the guidebook section, so a wrong answer is traceable to a wrong source instead of being a mystery.
- The corpus is plain markdown in the repo, so updating a fact is a pull request, not a prompt rewrite.
What I would do differently
Chunking by headings was the right call, but I would add an evaluation set of real questions from last year’s students before touching the prompt again. Retrieval quality sets the ceiling; the model only decides how nicely the ceiling is described.