Step 5 – Prepare photos

1. Understand collection of portraits need

For best results, OpenCV-train function need a collection of portraits with these properties :

  • picture shows face
  • picture is gray-scaled
  • all pictures have same size
  • eyes are horizontally-aligned
  • at least 30 pictures per people (my own experience)

See Han Solo’s picture as a good exemple :
hs1
You could create all this pictures with Gimp… but it’ll be a painful job.
To do it for you, I wrote a C++ software, strongly inspired from Philip Wagner Python script (here). This software, called prepare do, for each jpg file found in current directory :

  • resize the picture (for performance issue : your Pi is not powerfull enough to work with 5M pictures) (target dimension is a parameter, I recommend 800px width)
  • transform to gray scale
  • equalize colors (optional, parameter)
  • detect face
  • detect eyes (with glasses or not)
  • rotate picture to align eyes
  • crop picture around the face (size is an input parameter)
  • create a new picture sizexsize pixels (size is a parameter)
  • save new picture using a prefix to name it (prefix-index.jpg)
  • Warning : only put in input photos with only 1 person. Anyway, only the first person face will be cropped.

./prepare 0.3 100 hs 800 1

  • 0.3 means 30% = I take 30% more than the distance eye-eye.
  • 100 means : my output picture will be 100×100 pixels
  • hs means :my ouput picture name will start by hs (hs1.jpg, hs2.jpg, hs3.jpg…)
  • 800 means : original input pictures will be resized to 800 pixels width (ok for RPI performance)
  • 1 means : do the color histogram equalization (0 = don’t)

With these parameters and with the Han Solo’s picture below, you ‘ll get the gray small face displayed before.

hansolo

2. Download source code

Download .cpp source code here. http://raufast.org/download/preparePhoto.cpp

3. Compile

CMakeLists.txt should contain :

cmake_minimum_required(VERSION 2.8)
project( prepare )
find_package( OpenCV REQUIRED )
add_executable( prepare preparePhoto.cpp )
target_link_libraries( prepare ${OpenCV_LIBS} )

Then, compile :
cmake

make .

And run within a directory full of pictures !
Warning, this soft doesn’t recognize all faces. Especially if original face is tilting too much. My recommandation is to provide a lot of input pictures and see how many you get at the end…
If everything is ok, you should have dozens of nice 100×100 portrait. Perfect for your face recognition training, next step.

At this stage, you should do a break and read The Man Who Planted Trees (Jean Giono).

Step 6 – Face Recognition !

1. Let your Pi talk !
Now, your Pi must be able to speak. A very easy way to do it is to install espeak. (espeak is enough now, you don’t need to install Mbrola any more).
sudo apt-get update
sudo apt-get install espeak

espeak

Test it using a simple command line
espeak “hello world.”
Voice by default is english. If you want to change, select your voice using -v flag.
List of languages is here.
espeak -vfr “coucou lapin.” will talk with french accent.
For same language, you can select alternative voices using +m1 (for male voice, 1..4) or +f1 (for female voice, 1..4).
Speed is also a parameter : -s130 (by default : 100).
I recommend you to play with all these parameters to find a not-too-much-creepy voice 😉
If you want to read content of a file, just use -f option.
espeak -f “to_read.txt” -vfr+f2 -s130

2. understand source code
Once again, this source code is strongly inspired from Philipp Wagner source (here).
Download the source code here. I added a lot of comments to be more pedagogic as possible.
Algorithm is quite simple :

  • read config files with training pictures previously created in step 6 : store them in a picture collection
  • init some parameters, create a Eigen model (could be a Fisher model also)
  • train the Eigen model with collection of pictures
  • init webcam and start an infinite loop
    • capture photo
    • detect faces
    • for each face
      • try to recognize people leanrt
      • if success (confidence > threshold)
        • put name in the output image
        • speak to this person
        • display the output image

All funny parts are inside the speakTo function. Here you can add as many smarts behaviors as you want. It’ll let think your mirror is really a magic one.
Some exemples :

  • say time
  • depending hour of the day, propose important things to do : propose a morning coffee, remind to wash teeth before to go to sleep, remind to call his mother all sunday, ..)
  • connect to internet to get weather, stocks exhange, google calender next meeting, (see step 7 of this blog for code exemples)
  • adapt your speak to personn recognize (basic rules: tell a woman she’s beautiful, and tell a man he’s strong).

For such “smart agent” : the Wii balance board is a very good exemple : when you connect, your Wii always tell you short, funny and very-well adapted things…

3. compile
This step should not be difficult if you use this CMakeLists.txt. (notice the libopencv linked as already seen in step 3)

cmake_minimum_required(VERSION 2.8)
project( reco)
find_package( OpenCV REQUIRED )
add_executable( reco faceReco.cpp )
link_directories( /home/pi/pierre/libfacerec-0.04 )
target_link_libraries( reco /home/pi/pierre/libfacerec-0.04/libopencv_facerec.a ${OpenCV_LIBS} )

4. Test & Enjoy
Now, it’s time to test and to enjoy !
./reco csv-filename histo threshold
./reco faces.csv 1 5500

  • csv-filename : filename of csv file where you put all your training pictures path
  • histo : 0/1 : do you want to do a color equalization histogram before detecting face ? (try both and decide)
  • threshold : level of confidence above wich, you decide to recognize somebody (need to be tune after first run : use 5000 by default for Eigen Model (less for Fischer model)

Note : Light sensibility will depend on your webcam. With my logitech C270, it works well with natural light and artificial light.
Play with paramater (including training picture creation) and find your best set of parameter.

5. Install your Magic Mirror !
Select a wooden home-made mirror, put the webcam on its top.
Put your Pi and speakers behind the mirror to hide all technical stuffs.

mirror02

At this stage, you should have a magic mirror, which is a perfect geek useless stuff. Start to impress your girlfriend, your step-mother or the pizza deliveryman.

Step 7 – Smart Mirror Behavior

1. Gran Torino syndrom

In the video, the mirror reminds me my next meeting. This smart behavior is done connecting our software to google calendar thru API.
Frankly, I don’t understand why Google didn’t make C++ API to connect to Google Calendar. I’m feeling like Clint Eastwood in Gran Torino, with my old good gcc compiler, alone and abandoned by all.
clint

2. Using Libgcal

Hopefully, there is Ligcal, a C library to handle google calendar and contacts. I don’t know if it’s officially supported.
sudo apt-get update
sudo apt-get install libgcal0
sudo apt-get install libgcal-dev

3. Testing Libgcal

This small source code (download here) connects to google calendar (thru login/pwd given as arguments or harcoded), gets all yours events/meetings and selects the next one.
gcc -lgcal testcal.cpp -o cal

Frankly, it doesn’t work each time. Curiously, this code works randomly. It get connected but sometimes, it doesn’t return list of events :-(. If you find why, please let me know.

gcalender

Once tested, copy this exemple to your mirror source code, create a dedicated function and call it from the speakTo function.

At this stage, you should be able to have a smart mirror which reminds you your next meeting. Create a new event “do a backup” and let your mirror do its job…