Great article, as usual. I have a question about the tracing bit though - some clients are very sensitive about their data and wouldn't really like anyone to be able to look at it in the trace (input or output). How do you deal with this situation? Deploy something (e.g. Opik) in-house?
If you want to go all the way, yes, you can deploy tools like Opik on-prem. But they also support a multi-tenant architecture with permissions, so only the organization and authorized users can access the data.
Have you come across a case where the data was private user data ( E.g apps analysing health reports - chatgpt, Microsoft etc come to mind)? I'd imagine even admins wouldn't ideally be allowed access
“Next time you see a vendor dashboard with dozens of RAG metrics, map each one back to the six relationships. If an indicator does not clearly measure one of the core links, it is noise. Drop it and focus on what actually diagnoses failures”
One of the key impressive linking concepts was the Maslow hierarchy style of visual representation. Plus, I liked the pop references sparsed in the contents. Thank you for the read!
Great article, as usual. I have a question about the tracing bit though - some clients are very sensitive about their data and wouldn't really like anyone to be able to look at it in the trace (input or output). How do you deal with this situation? Deploy something (e.g. Opik) in-house?
If you want to go all the way, yes, you can deploy tools like Opik on-prem. But they also support a multi-tenant architecture with permissions, so only the organization and authorized users can access the data.
Have you come across a case where the data was private user data ( E.g apps analysing health reports - chatgpt, Microsoft etc come to mind)? I'd imagine even admins wouldn't ideally be allowed access
“Next time you see a vendor dashboard with dozens of RAG metrics, map each one back to the six relationships. If an indicator does not clearly measure one of the core links, it is noise. Drop it and focus on what actually diagnoses failures”
real awesome takeaway here. great read thanks :)
Thanks, man!
One of the key impressive linking concepts was the Maslow hierarchy style of visual representation. Plus, I liked the pop references sparsed in the contents. Thank you for the read!