Face2Gene and syndrome recognition by computers- photographs needed, try Facebook?

Face2Gene (F2G) is a new company which aims to use computer-based facial recognition rapidly to analzye large numbers of clinical photographs of children and adults with syndromes of genetic aetiology.  The faces of individuals with such syndromes can be highly recognizable, and the ability to recognize them is a skill which can be learnt. The ability to do this is not restricted to members of the medical profession, still less Clinical Geneticists, and it appears that computers are good at it too.

The face of a child with Down syndrome is probably the most recognizable 'syndrome face'.  Here is another example, taken from a journal article: 

Reproduced from: Koolen D et al Journal of Medical Genetics (2008) 45:11 710-720.  Freely available here

The individuals shown have a condition called 17q21 microdeletion syndrome. Their resemblance to each other is (mostly) striking; indeed, they resemble each other far more than they resemble members of their own family, which is amazing when one considers how tiny is the genetic change resulting in the syndrome in question.

It is possible to study the pictures to gain an idea of which individual bears the closest resemblance to the 'most typical'.  I would go for numbers 7, 14 and 20, with numbers 1, 3 and 11 being perhaps the least typical.  This categorization is the kind of thing that Clinical Geneticists do and it has been immensely valuable in delineating new syndromes.  

Clinical Geneticists usually take photographs of undiagnosed patients attending their clinics, with consent, and these photographs are used as an aid to diagnosis in the way outlined above. Although Clinical Geneticists take and store photographs of these children, they do not ultimately have ownership of the clinical photographs of the patients in their collections- the parents do; and, if they have capacity, ultimately the affected individuals themselves do.

F2G have created an app for use by Clinical Geneticists, in which clinical photographs of patients with undiagnosed conditions are uploaded to their website and analyzed by computer to generate a list of possible 'syndrome matches'.  Their motivation in doing this is not purely an altruistic one of helping Clinical Geneticists diagnose their patients, but also one of making the data available to pharmaceutical companies who are developing therapies to treat children with rare disorders.

On first trying out the app, I experienced simultaneously feelings both of excitement and disappointment. Excitement because I think that this has the potential to be very useful for my specialty of Clinical Genetics; disappointment because I was immediately struck that something seemed to be missing, and that was immediate access to a well-catalogued library of clinical photographs.  

As  the Clinical Geneticists on the  advisory board of F2G will perhaps have advised, access to photos of patients with known, diagnosed and molecularly confirmed syndromes is a powerful training resource for members of our specialty. Given that Clinical Geneticists spend a lot of time looking at facial photographs, they need good resources for training in this difficult skill, a skill which will still be needed even when the patient's entire genome sequence is available (see last month's post).  I believe that F2G could -and should- help to provide such a resource. 

We are told that F2G has a database of over 150 000 photos, clearly a very extensive resource.  Why are these photos not made available to Clinical Geneticists through the F2G app as a training resource?  Presumably it is because they do not have consent to release these photographs in this way. A pity.

Of course, there are plenty of training resources available to Clinical Geneticists already, including many text books and electronic databases, such as the London Dysmorphology database.  The latter is comprehensive and useful, but does suffer from some disadvantages, which the authors themselves would I am sure concede: it cannot make any photo available without consent, either directly from the patient or from the journals who own the copyright and who are therefore authorized to make them available to LMD.   The number of photographs is therefore limited.  Second, they are not always of the best quality.  Third, they are often a mix of different types of photo- different parts of the body, radiographs, MRI images and so on.  

In order to increase their resource still further, F2G have embarked on a quest to ask Clinical Geneticists to make clinical photographs of patients with molecularly confirmed syndrome diagnoses available to them.  Again, as I understand it, this is on the basis that the photographs will not be made available to the Clinical Genetics community, but will simply be 'read' by the face recognition software and thus used only as an aid to machine learning.  

More photos are surely better.  It must be a basic principle of computer-aided recognition that a machine will learn more from 1000 photos than from 10 or 100. Increasing the amount of data should improve recognition accuracy and might even enable detection of different 'subtypes' of face within a given syndrome; or allow for discovery of which part of the face is most characteristic.  It should also in theory allow for better correlation between genotype and facial phenotype, if such correlations exist.

I don't think that F2G will get much engagement from Clinical Geneticists in terms of gaining access to clinical photographs unless there is some reciprocal benefit. It would make sense if photographs from archives of individual Geneticists could be made available as a public resource for training purposes as outlined above. Each Geneticist must have dozens if not hundreds of such photographs, which, if pooled on a national or international basis, would constitute an amazing resource for practising clinicians.

Sadly, making such a resource available is not in prospect because Clinical Geneticists are not authorized to release their photographs in this way- they don't have consent from the individuals and families concerned.  They might possibly be able to allow their archived photographs to be read by computer to promote machine learning, but it's not clear what the trade off for them in doing this would be.

To increase access to photos by Clinical Geneticists, here is an idea.  We have all been noting the growth of the involvement of patients and families in social networking sites, especially Facebook.  I have been a member of a couple of sites dedicated to specific syndromes for a while, and I have noted that they attract decent numbers of followers, worldwide.  The love for and dedication to their children that the parents have is plain for all to see.  They are a self-selected group of course but their willingness to share information to help each other is clear.  Research projects into the conditions from which their children suffer are highlighted in detail.  Photographs are shared: I was struck by one post in which a parent put up a photo of her affected child and asked other group members to add photos because she wasn't sure what individuals with that syndrome were supposed to look like.  About 50 photos were added.

Why not cut out the middlemen (Clinical Geneticists) and go straight to the people who can decide about these photographs- the families themselves? Would they be willing to share photographs of their children with Face2Gene?  If evidence of potential benefit to them could be provided, then I think that the answer would be ‘yes’.  It would be incumbent on Face2Gene to explain what this benefit would be, but if what they are saying about their business plan is accurate, they should be able to do that.

These families are used to uploading photographs of their children to websites.  They have done it enough times on Facebook, and, if they thought that it would help their child or other children with the same disorder in the future, then they could easily do it on Face2Gene as well. The Facebook group as a whole could be offered a financial inducement (why not?).  In this way, a larger number of consented photos than Clinical Geneticists could ever provide would be made available, and direct engagement of patients and families would have happened- always a good thing.

If it became known that the support groups on Facebook were becoming actively engaged with clinical, research and commercial partners, then this would improve their standing and significance and lead to better engagement of affected families- the support groups would grow.  There would presumably be a clear win for Face2Gene as they would get the photos they wanted, meaning that their computers would have improved opportunities for learning.  For clinicians, there would be a gain in terms of improvement in educational/training resources, as the photographs would now have the level of consent required to make them publicly available.  

With all this training and learning happening, I'd be interested to see whether computers could 'beat' Clinical Geneticists at syndrome recognition.  I wouldn't bet against them.