Zet / open-source Python
Zet sends uncertain Laya answers to human review.
Zet adds a sure or unsure decision to Laya's answers about text. It uses examples with known correct answers to decide which future answers can be used automatically and which should be checked by a person.
View the Zet repository on GitHub →
What we measured
On 350 human-labeled voice-assistant requests with six possible topics, Laya alone gave 45 wrong answers. Zet sent 41 of those mistakes to review while accepting 227 answers automatically. Four wrong answers still went through.
Zet sent 123 answers to review in total, including 82 that Laya had right. No person performed those reviews in this test. The result shows a useful tradeoff on this dataset; it does not establish the same result for your emails.