[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"$f12frelygqk1kk":3},{"_id":4,"slug":5,"title":6,"subtitle":7,"kind":8,"cards":9,"tags":18,"categories":20,"source":22,"lang":25,"author":26,"audioState":29,"stats":30,"publishedAt":33,"renderer":34},"6abc886fca21c797c7ea011f","zet-an-open-source-layer-on-laya-that-knows-when-to-ask-a-hu-de8d568f","Zet: an open-source layer on Laya that knows when to ask a human","Confidence scores aren't error rates.","news",[10,13],{"headline":6,"body":11,"imageUrl":12,"sourceImageUrl":12},"Confidence scores aren't error rates. Zet sits on top of Laya and marks every answer sure or unsure, using conformal prediction sets and a Learn-then-Test threshold calibrated per language, so the answers it automates stay within an error budget you choose (say 5%). On MASSIVE (human-labeled, 300 examples per language, 5% budget), it automated 84% of English and 80% of Swedish requests on a 6-scenario task, with 0 errors in 75 sure English answers. When the task got harder (18 scenarios), it automated nothing, while its prediction sets still contained the right answer 97-99% of the time. That refusal is the feature working. Runs locally on ONNX Runtime with no PyTorch, with a web interface for creating tasks, uploading labeled examples, calibrating, and reviewing unsure answers. Apache-2.0, built on Laya (not endorsed by its authors).","https:\u002F\u002Fmedia2.dev.to\u002Fdynamic\u002Fimage\u002Fwidth=1200,height=627,fit=cover,gravity=auto,format=auto\u002Fhttps%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Foetn0tk0a06kjduzw9my.png",{"headline":14,"body":15,"imageUrl":16,"images":17},"For further actions, you may consider blocking this","For further actions, you may consider blocking this person and\u002For reporting abuse","\u002Fapi\u002Fmedia\u002Fposts\u002Fzet-an-open-source-layer-on-laya-that-knows-when-to-ask-a-hu-de8d568f\u002F1.webp",{"local":16},[19],"dev",[21],"Technology",{"name":23,"url":24},"Dev.to","https:\u002F\u002Fdev.to\u002Fyoosseph\u002Fzet-an-open-source-layer-on-laya-that-knows-when-to-ask-a-human-1o6i","en",{"handle":27,"displayName":28},"spots","Spots","queued",{"views":31,"likes":32,"saves":32,"shares":32,"completions":32,"opens":32,"skips":32,"depthSum":32},1,0,"2026-09-30T03:56:31.900Z","local"]