Reducing bias in eyewitness sketches

This commentary is part of a series in TODAY’s Science section, in collaboration with the National University of Singapore’s (NUS) School of Computing, which explores the computer science research projects conducted here.

In 1985, Kirk Bloodsworth was convicted for the assault, rape and murder of a nine-year-old victim in Maryland, United States, and sentenced to death. The testimonies of five witnesses overwhelmingly pointed to him. But they were wrong. Bloodsworth was innocent, and it took nine more years for the then-novel DNA forensics technology to exonerate him, and to release him from prison.

Sadly, such cases of misidentification are common. Since 1989, an estimated 75 per cent of wrongful convictions in the US exposed by DNA testing were due to eyewitness misidentification. This confirms more than 30 years of psychological studies showing that witness testimonies are highly unreliable.

To be sure, changes have been made to the protocol of obtaining eyewitness testimony, including the eliciting of the face sketch, to improve its reliability. Modern software tools are used to create a composite face sketch, by mixing and matching pre-defined face parts such as the eyebrows, nose shapes, lips, and jawline. But some of the fundamental problems of eyewitness recall, such as personal bias and memory malleability, remain unsolved. These problems result in the generation of face sketches that do not resemble the actual person.

So, why are eyewitness sketches still being used by police worldwide? A major reason is the lack of a viable alternative. And we, at the NUS School of Computing, hope to provide one.

My recently graduated student, Dr Hossein Nejati, and I seek to combine psychological insights with image analysis algorithms to tackle this challenging problem. By studying how humans perceive and remember faces, we have devised a novel method to first elicit the face sketch from a witness’ memory, and then to match the sketch against a criminal database of photographs. Our method employs three novel ideas.

The first, and most radical, idea is to get the witness to draw the sketch himself. The existing method requires the witness to describe the face verbally to an artist, who then draws the sketch. By removing the middleman (the artist), our method thus removes any bias the middleman may inadvertently introduce into the sketch. Moreover, since the witness does not verbalise the face, our method bypasses the verbalisation step, which is known to be flawed as it does not match how faces are remembered in the brain.

To help witnesses draw better, we also intend to develop an iPad sketching app that provides a generic face for the witness to modify so that he does not have to draw from scratch. Our app will also constrain the proportions of facial parts, to avoid grotesque faces.

The second idea is to remove the witness’ personal bias. Psychological studies have revealed this quirk: You and I can look at the same face, but we will remember it differently. The way we remember faces in our brain depends on our individual history of the faces we have encountered in the past. Thus, I may remember a large nose, but in reality the nose is not unusually large; it is large only when compared with those I previously encountered.

To learn the personal bias, after the witness has drawn the sketch of the suspect from memory (called the Initial Sketch), he then draws sketches of three or four other photographs. This second set of sketches reveals his personal bias. Our method then uses this information to compensate the Initial Sketch, so as to produce one that is (hopefully) less biased.

Finally, the third idea in our method is to mimic how the brain encodes faces to compare the sketch against the database of photographs.

This is done via a computer program we developed, based on a psychological model, called the Exception Report Model, which posits that the brain remembers differences rather than the entire face image. That is, to remember a face, the brain computes the difference between it and the average of faces previously encountered (the personal bias).

Differences in the shapes and colour of face parts, e.g. the nose, eyes, and lips are encoded in the brain. This “difference encoding” captures how “exceptional” the given face is, when compared with the average. It also explains why caricatures work: Caricatures of US President Barack Obama typically show him with large ears and an exaggerated jaw because these are precisely where his face deviates from the average.

We have run extensive tests on our eyewitness sketch-and-match method — on more than 700 sketches of faces from various ethnic groups, including Caucasian, Indian, Middle Eastern, African, Latino and East Asian — and we are happy with its performance. Our next step is to do a trial with actual witnesses. While still far from a ready product, we are hopeful that our method will save innocent people such as Bloodsworth from the agony of a wrongful conviction.

About the Author:

Dr Terence Sim is an associate professor at the School of Computing, National University of Singapore. He leads a team of researchers in the analysis and modelling of facial images, and also conducts research in biometrics and computational photography.