Imagine if we place a screen in a hospital room that can be used by different parties, and for each person or role we show a different screen with interaction possibilities or we create mobile applications that are used by different healthcare providers, based on the facial recognition of the person. The only thing we need are 3 pictures of the person to train the model.

For this I created a proof of concept that is built with the help of Azure Cognitive Services, more specific the Face Api.
The 2 main features of this service are:

  • Face Identification
  • Face grouping.

With these tools I developed a web dashboard that is connected to a webcam, based on your face,  the screen shows custom tailored information that is suited only for your role.

There are 3 groups defined in the proof of concept:

  • Doctors
    • Can see detailed information of the patient metrics
    • Knows what’s happing at a glance, no need to login or move between different systems
    • Has access to different action buttons
  • Nurses
    • Can see the needed medical information of the patient
    • Overview of all the medical handlings.
    • Has access to different action buttons
  • Patients
    • Has access to different action buttons
    • Can see general information (cafeteria information)
    • Overview of their care team.
    • Knows when certain healthcare providers are coming

Some screenshots!:

The initial screen scans your face via the webcam

If you are recognized as a doctor, following screen will be visible:

If you are recognized as a nurse, following screen will be visible:

If you are recognized as a patient, following screen will be visible:

 

Here you can see it in action:

 

This video shows how we train the model and add a new face.

 

We need to keep in mind that this feature alone can not be used as an authentication method. The face api only analyses images, thus someone showing a picture in front of the webcam would be recognized as this person. But we can enhance this functionality with voice recognition or sms verification, see it as multi factor authentication. This solution could also be used as an authorization provider after the authentication has happened