Sirma Computer Vision Lab has performed series of validation tests aiming to score the accuracy of the algorithms used in our leading facial recognition technology - MarketVidia. We have obtained excellent results against the benchmark Microsoft Azure FaceAPI. The accuracy of Sirma technology is 95.212%, versus 87.5752% accuracy of Microsoft Azure FaceAPI.
What we have done so far?
We have prepared a test set of 398,428 pairs of pictures of people - in half of them, in the pair of pictures the person is the same, and in the other half - the person is different.The whole set is verified by people (not software), which guarantees 100% accuracy to the test base. The pictures are downloaded from different sources on the Internet, they have got different quality, size and resolution, and are not processed additionally (they are close to real life). In preparing the test set, we took special measures to exclude pictures being used in the training of our models.
With this test set, we examined the patterns already trained, funding for each of them to the threshold value that minimized the error. The best outcome to date is 377,863 correctly made comparisons between the two faces (matching result with human assessment - “the same individual” or “different people”), thus, we achieved an accuracy of 94.8385%.
We compared the same test set against the leading technology used in Microsoft Azure FaceAPI. The pairs of pictures in which both faces were recognized were 365,447 (there were a significant number of images - about 8.3% where Microsoft technology could not find a human face at all), out of which 347,952 were correctly identified, according to the above method. The result obtained is 87.5752%. The relatively high number of undetected faces is caused by the permissible rotation of faces for Microsoft Face Detector, which is less, there are higher requirements for a minimum size of the face, and in addition the faces whose parts are overlapped with something else are not counted. If we look only at the pairs where Microsoft Face API was able to recognize faces, the results are: 95.4473% accuracy of Microsoft, compared to 95.2127% accuracy of our model.
This achievement of Sirma Computer Vision Lab team is the next step in validating of our face recognition technology, and give users a strong advantage in its implementation in various commercial applications.