Onfido LogoOnfido Logo

Developers

Facial Similarity reports

Start here

This guide presents a technical overview of Onfido's Facial Similarity report.

Introduction

Onfido's Facial Similarity reports compare the most recent live photo, live video or motion capture provided by an applicant during an identity verification flow to the face on their most recently captured identity document (such as passports or national ID cards). Where the document has two sides, Onfido will search both sides of the document for a face. Facial Similarity reports are designed to prove identity document ownership, so that only the owner of the identity document can use it to verify their identity and access services.

Onfido offers four types of Facial Similarity report: Photo, Photo Fully Auto, Video and Motion.

Photo and Photo Fully Auto

In Facial Similarity Photo and Photo Fully Auto reports, images and data are extracted from identity documents using machine learning, and then compared to a live photo taken and submitted by the applicant during their identity verification journey. The report assesses whether the holder of the identity document is the same person as the one on the identity document, while also ensuring the selfie image is a live capture taken by the applicant at the time of submission.

The Photo report uses a combination of algorithms and expert manual review by Onfido's super recognizer analysts to return a result.

The Photo Fully Auto report is intended to be paired with manual review by your own team, and does not include expert review by Onfido's super recognizer analysts. This guarantees faster results, but it also means more reports could potentially end with faces not being detected, or with users incorrectly rejected.

Photo and Photo Fully Auto API response breakdown tree

The result for both Photo and Photo Fully Auto reports can be clear or consider, returned in the output object of a Studio workflow run. The report result is determined by the results of individual breakdowns and sub-breakdowns.

The following diagram shows how these breakdowns and sub-breakdowns are mapped to particular results:

Face Photo breakdown tree

Breakdown descriptions

BreakdownSub-breakdown(s)Description
Visual authenticityAsserts whether the person in the live photo is real and not a spoof.
Spoofing detectionContains a score value between 0 and 1 which expresses the likelihood that the live photo is real, where 0 represents a spoofed image and 1 represents a real image.
Face comparisonAsserts whether the face in the document image matches the face in the live photo.
Face matchContains a score value between 0 and 1 which expresses how similar the document image face and live photo are, where 1 represents a perfect match.
Image integrityAsserts whether the quality and integrity of the uploaded files were sufficient to perform a face comparison.
Face detectedAsserts that a single face of sufficient quality was found in both the document image and the live photo.
Source integrityAsserts whether the source of the live photo is trustworthy. For example, it has not been digitally tampered with or was not taken from a fake webcam. This sub-breakdown also asserts whether the applicant's document is issued by a country subject to comprehensive US sanctions.

Complete details regarding the structure of Facial Similarity Photo and Photo Fully Auto reports and example results can be found in our API reference documentation.

Motion

In addition to extracting images and data from identity documents, Facial Similarity Motion provides added security for high-risk users or transactions, in a fully automated fashion. In addition to confirming a user's face matches their document upload, Motion also assesses liveness by asking users to complete a simple head turn pattern in both directions.

Motion API response breakdown tree

The result for Motion reports can be clear or consider, returned in the output object of a Studio workflow run. The report result is determined by the results of individual breakdowns and sub-breakdowns.

The following diagram shows how these breakdowns and sub-breakdowns are mapped to particular results:

Face Motion breakdown tree

Breakdown descriptions

BreakdownSub-breakdown(s)Description
Visual authenticityAsserts whether the person in the motion capture is real (not a spoof) and live.
Spoofing detectionContains a score value between 0 and 1 which expresses the likelihood that the motion capture is real, where 0 represents a spoofed capture and 1 represents a real motion capture.
Liveness detectedAsserts whether the head turn completion pattern was executed correctly.
Face comparisonAsserts whether the face in the document matches the face in the motion capture.
Face matchContains a score value between 0 and 1 which expresses how similar the face in the document image and motion capture are, where 1 represents a perfect match.
Image integrityAsserts whether the quality and integrity of the uploaded files and the content contained within them were sufficient to perform a face comparison.
Face detectedAsserts that a single face of sufficient quality was found in both the document image and the motion capture.
Source integrityAsserts whether the source of the live photo is trustworthy. For example, it has not been digitally tampered with or was not taken from a fake webcam. This sub-breakdown also asserts whether the applicant's document is issued by a country subject to comprehensive US sanctions.

Complete details regarding the structure of Facial Similarity Motion reports and example results can be found in our API reference documentation.

Video

Facial Similarity Video also provides added security for high-risk users or transactions. In addition to confirming a user's face matches their document upload, the report also assesses liveness by asking the user to record themselves repeating out loud 3 randomized digits displayed on their device screen and performing a randomized head turn movement.

The Facial Similarity Video report uses a combination of algorithms and expert manual review by Onfido's super recognizer analysts to return a result.

SDK localization

We recommend that you localize the strings if you're using one of the Onfido SDKs, to ensure that users understand the liveness head turn and speaking instructions.

By default, the Onfido SDKs detect the language to use from the user's device or browser settings. We recommend, however, you pass the code for the expected language to the locale mechanism, in any of the Onfido SDKs:

  • iOS SDK - pass the onfido_locale parameter
  • Android SDK - pass the onfido_locale parameter
  • Web SDK - pass the locale parameter

The following languages are supported for string localization:

Language nameLanguage code
English"en"
Spanish"es"
Indonesian"id"
Italian"it"
German"de"
French"fr"
Portuguese"pt"
Polish"pl"
Japanese"ja"
Dutch"nl"
Romanian"ro"
Basque"eu"
Catalan"ca"
Galician"gl"
Chinese"cn"
Turkish"tr"
Malay"ms"

Some string localizations are available out of the box, but this differs for each SDK.

You can also provide your own custom translations to your users.

Video API response breakdown tree

The result for Video reports can be clear or consider, returned in the output object of a Studio workflow run. The report result is determined by the results of individual breakdowns and sub-breakdowns.

The following diagram shows how these breakdowns and sub-breakdowns are mapped to particular results:

Face Video breakdown tree

Breakdown descriptions

BreakdownSub-breakdown(s)Description
Visual authenticityAsserts whether the person in the live video is real and not a spoof.
Spoofing detectionContains a score value between 0 and 1 which expresses the likelihood that the live video is real, where 0 represents a spoofed video and 1 represents a real video.
Liveness detectedAsserts whether the numbers and head movements were executed correctly during the live video task.
Face comparisonAsserts whether the face in the document image matches the face in the live video.
Face matchContains a score value between 0 and 1 which expresses how similar the document image face and live video are, where 1 represents a perfect match.
Image integrityAsserts whether the quality and integrity of the uploaded files were sufficient to perform a face comparison.
Face detectedAsserts that a single face of sufficient quality was found in both the document image and the live video.
Source integrityAsserts whether the source of the live photo is trustworthy. For example, it has not been digitally tampered with or was not taken from a fake webcam. This sub-breakdown also asserts whether the applicant's document is issued by a country subject to comprehensive US sanctions.

Complete details regarding the structure of Facial Similarity Video reports and example results can be found in our API reference documentation.

Facial Similarity Report task

Onfido Studio features workflow tasks for all four Facial Similarity report variants. When constructing a Studio workflow, a Facial Similarity Report task must always follow the path of a Document Capture task and a Face Capture task (which also has photo, video and motion variants) in order to extract the appropriate task input data to make the comparison.

It is also highly recommended that the workflow include a Document Report task.

Below you will find an illustrated example of a Studio workflow running a Facial Similarity report:

Facial Similiarity workflow

More information about report tasks can be found in our Studio Product Guide.

Facial Similarity Report task results

If you want to obtain the specific outputs from the Facial Similarity Report Task via the Onfido API (for example, the overall report result or breakdown results), you can manage this by configuring the Workflow Output in the Studio Workflow Builder. You can also retrieve all associated output data from a Facial Similarity report.

You can refer to our Studio product guide for more information on Workflow Output configuration. Once set up, the output can be consumed by making a Retrieve Workflow Run call to the Onfido API. Report results are found in the output property.

Workflow Run results can also be accessed on the "Results" tab of your Studio Dashboard.

Suggested client actions

In this section, we explore a range of specific scenarios for the Facial Similarity Report by examining individual results from the report breakdown and describe suggested actions. We provide suggestions for clients who manually review photos as part of their service, and those who do not.

This section also presents a table highlighting which Facial Similarity report breakdowns can be affected by user environments, and what advice you can pass on to your users.

The details of the Facial Similarity Report breakdown structure are described above in this guide, and are also documented in our API reference.

Example scenarios

Scenario 1: All results are clear

In this scenario, the overall report result will be clear and we recommend that you allow the user to proceed.

Scenario 2: face_detected result is consider

  • Ask user to retry
  • Make sure that both the face on the document and the face in the selfie are clearly visible
  • Confirm the document image with side labelled as front contains a face

Scenario 3: source_integrity result is consider

Type of clientContributing breakdown(s)Suggested client action
ALL clientsdigital_tamperingBlock user. Evidence has been found that the user deliberately used photo editing software to manipulate their selfie image.
ALL clientsfake_webcamBlock user. Evidence has been found that the user is running software capable of manipulating their webcam feed. This might be deliberate, or might be that the user installed the fake webcam software for other purposes and then forgot to turn it off. If you are not risk averse, consider asking the user to retry.
Clients WITH manual reviewtime_of_capture* If you are not allowing users to upload photos, then review the image. If the image looks genuine, ask the user to check date and time is correct on their device and retry.

* If you are allowing users to upload photos (not recommended): ask the user to re-upload a new, more recent selfie.
Clients WITHOUT manual reviewtime_of_capture* Ask the user to check date and time is correct on their device and retry.

Scenario 4: face_match result is consider

  • EITHER block the user OR ask the user to retry
  • Suggest the user retries with a more recent document

Scenario 5: spoofing_detection result is consider

  • EITHER block the user OR ask the user to retry

Scenario 6: liveness_detected result is consider

  • Ask user to retry
  • If you are not risk averse, and all other breakdowns are clear, ignore and allow the user to proceed

You should always cap the number of user retries in line with your tolerable business risk.

Breakdowns affected by user environment

The following table highlights which individual breakdowns in the Facial Similarity report can be affected by user environments, and what actions you can suggest to users:

Suggested user actionsface_detectedface_matchspoofing_detectionliveness_detected
Improve lighting conditionsXXX
Take both document and selfie images against a plain backgroundXXX
Avoid any glare caused by light sources in the backgroundXXX
Make sure the face is sufficiently close to the camera (on SDK, make sure the face fills the oval)XXX
Make sure the document is sufficiently close to the camera (on SDK, make sure the document fills the region defined by the overlay)XX
Take a frontal selfie, with a straight headXX
Make sure there are occlusions (e.g. glasses)XX
Have a neutral facial expressionX
Make sure your face is not blurryXXX
Do not have anyone else in the pictureXXX
Do not use filters or devices with overlaysXXX
Do not hold the document in the selfie imageX
Say the digits displayed on the screenX
Turn your head in the direction displayed on the screenX