Smile detection with OpenCV, the “nose” trick

Object detection and recognition on pictures was the very first door I got through into the fascinating world of machine learning and artificial intelligence. That was many years ago. Back then, I worked on face detection and recognition project and I discovered what I consider as one of the most cutting-edge algorithms in machine learning: the Viola and Jones’ object detection algorithm.

The algorithm, initially designed for face detection, has been used with a variety of objects, including buildings and cars.

Some time ago, I tried OpenCV’s integrated cascade model for  smile detection  just for fun and to see how it works. I was a bit disappointed because of the huge number of false detections (smile detections  where there is no smile or no face at all). My first thought was naturally to detect faces on a picture and then to feed the smile detector with the detected faces.

This solution fairly reduced false alarms caused by the smile detector.  unfortunately my glasses were  still detected as two smiling mouths, not to mention false alarms  generated by the face detector itself.

I then had a look at OpenCV’s available cascade models, and guess what, there is a rather good model for nose detection. Nobody would argue that the mouth is located under the nose. This is why my final recipe, which finally achieved the goal of greatly limiting the number of false alarms, is the following:

  1. Detect all faces on a picture
  2. Detect the nose on each face and discard faces with no nose
  3. If many noses are detected on one face, choose the lowest one as a split point
  4. Try detecting a smile on the lower half of the face starting from the center of the nose


There is no better than a piece of code to illustrate the above recipe:



Leave a Comment

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s