Metamaze
Search…
Fundamentals
Human validation
The human validation module in Metamaze allows for human control after prediction of an AI model. In Metamaze, there are two steps in which an AI model performs a prediction
- Document creation: the model predicts which pages in a uploaded file belong together and together form a document of a certain document type. It splits an uploaded file into different documents (page management model) and predicts the document type and the language of each individual document (document classification model). If the page management model is not used, we only talk about document classification.
- Entity extraction: the model tries to recognise information (one or more words, also called entities) from the document.
Each prediction is accompanied by a prediction score. The document creation model predicts by x% certainty that a document belongs to a certain document type. The entity extraction model predicts with x% certainty that one or more words in the document is an entity.
If human validation is enabled, an upload of a series of files or documents in this case if certain conditions are met:
- The document creation and prediction certainty score is lower than the set threshold for a certain document type.
- The entity prediction collateral score is lower than the threshold set for that entity
- An entity set as 'required' was not found
- An entity could not be converted to a particular format correctly. E.g.: the AI model finds a value for the entity of type 'date' that does not represent a date. The system tries to convert the text to the date of a certain format and fails.
However, it is possible to enforce human validation at each step as an additional quality control.
All documents treated in human validation will be automatically sent to training with status 'to verify'. This allows for a re-training of the model on this documents however don't forget to create a suggested task on those to promote it to golden training data (status 'to verify' -> 'done'). Checks on quality are important, the model will be impacted by mistakes.
Last modified 8mo ago
Copy link