Discussion about this post

User's avatar
Alexey Grigorev's avatar

Thanks for having me, it was great to be able to demo the FAQ!

JP's avatar

Alexey's leave-one-out trick for FAQ evals is a neat fix for the cold-start problem. The curation step that returns NEW, UPDATE, DUPLICATE or WRONG_COURSE is also a natural fit for a decision model rather than a chat model: fixed labels, a confidence score, and your code decides what happens next. I've been collecting guides and experiments for that pattern on JevMade.

7 more comments...

Ready for more?