Face Aware OSD Photo Frame

This Instructables shows how to make a photo frame with face aware On Screen Display (OSD).

The OSD can show time, weather or other internet information you want.

Step 1: Why OSD Photo Frame?

I have 2 photo clock projects at Instructables before:

https://www.instructables.com/id/ESP32-Photo-Clock...

https://www.instructables.com/id/Arduino-BiJin-ToK...

Both retrieve a photo with beauties and a time board from the Internet every minutes and display on the LCD.

Displaying beauties is good, but they all are strangers to me. How about using personal favorite photos and add current time and further instant information on it?

This project is exploring how to make it.

Step 2: Why Face Aware?

Let's check how to add instant information OSD to a photo first:

1. Random select a photo from a specific folder
2. Retrieve time
3. Retrieve instant information from Internet
4. draw time and instant information on the photo

Step 1-3 is straight forward; Step 4 also look simple, but determine where to draw the text is not so easy.

If the text size too small, it is hard to read in a reasonable distance; If the text size too big, it most likely cover the photo objects. Especially if it is portrait photo, text covered the faces is not preferred.

Since the position of faces for each photo not the same, to avoid OSD covered faces we need a face detect process first. Then we can find a no-face-area to draw the text.

Step 3: 2-tiers Design

The face detect process require some processing power, in contrast, photo frame can be very light weight. So I split it into 2-tiers:

Server

The face aware photo engine is a Node.js app server. For each HTTP request, it will:

1. Select a photo from the photo folder randomly
2. Face detection
3. determine no face or least faces area
4. At the mean time, retrieve weather or other useful instant information from the Internet every certain period
5. Draw time and instant information on the photo
6. Return the photo with OSD in JPEG format as HTTP response

Client

The client can be a web browser, an applet or an IoT device.

E.g. an ESP32 dev boasrd with a 2-4 inches LCD is very fit for placing on the desktop as a tiny photo frame.

Step 4: Setup Photo Server Option 1: Docker Image

For convenience, I have pre-builded a Docker image for the face aware photo OSD Node.js app server.

In case you are not yet setup Docker, please follow the Docker Get Started guide:

https://www.docker.com/get-started

Then run the following command: (replace /path/to/photo to your own photo path)

docker run -p 8080:8080 -v /path/to/photo:/app/photo moononournation/face-aware-photo-osd:1.0.1

Test it by browse to http://localhost:8080/

You may find the displaying time is not in your time zone:

docker run -p 8080:8080 -e TZ=Asia/Hong_Kong -v /path/to/photo:/app/photo moononournation/face-aware-photo-osd:1.0.1

If you live in Hong Kong like me, you can add Hong Kong weather information:

docker run -p 8080:8080 -e TZ=Asia/Hong_Kong -e OSD=HK_Weather -v /path/to/photo:/app/photo moononournation/face-aware-photo-osd:1.0.1

If you would like to develop your own OSD information:

mkdir -p ~/git
cd ~/git
git clone https://github.com/moononournation/face-aware-photo-osd.git
docker run -it -p 8080:8080 -e TZ=Asia/Hong_Kong -e OSD=HK_Weather -e DEBUG=Y -v /path/to/photo:/app/photo -v ~/git/face-aware-photo-osd/app.js:/app/app.js moononournation/face-aware-photo-osd:1.0.1

Modify update_osd() function in app.js to tailor-made your own OSD information. After development, simply remove DEBUG=Y environment from the docker command.

Step 5: Setup Photo Server Option 2: Build From Source

If you are familiar with Node.js, you can build the app server from source.

Get the source:

git clone https://github.com/moononournation/face-aware-photo-osd.git

Install packages:

cd face-aware-photo-osd

npm install

Create photo folder and copy your own photos to the folder.

Run app server:

node app.js

Step 6: Client Option 1: Web Browser

Simply browser to http://localhost:8080/

The page is scripted auto load a fit page size image every minute.

P.S. If you browse from another machine that are not running the app server, remember change localhost to the app server hostname or IP address.

Step 7: Client Option 2: ESP32 + LCD

A photo frame client can be as simple as a ESP32 dev board and a LCD.

Here are the hardware required:

ESP32 Dev Board

Any ESP32 dev board should be ok, this time I am using a board called MH-ET LIVE.

LCD Display

Any Arduino_GFX supported LCD, you may find currently supported display at GitHub readme:

https://github.com/moononournation/Arduino_GFX

Jumper Wire

Some Jumper Wires, depends on the dev board and LCD pins layout. In most case 6-9 female to female jumper wires are enough.

LCD Stand

Some support help the LCD standing straight, this time I am using a card holder stand.

Step 8: ESP32 + LCD Assembly

The ESP32 with pin header on the upper side is preferred. If the pin header at the lower side, simply put the board upside down ;>

Connect ESP32 and LCD with jumper wires, then fit it to the stand.

Here are the sample connection summary:

ESP32   -> LCD
Vcc     -> Vcc
GND     -> GND
GPIO 5  -> CS
GPIO 27 -> DC (if available)
GPIO 33 -> RST
GPIO 18 -> SCK
GPIO 19 -> MISO (optional)
GPIO 22 -> LED (if available)
GPIO 23 -> MOSI / SDA

Step 9: ESP32 + LCD Software

Arduino IDE

Download and install Arduino IDE if you are not yet do it:

https://www.arduino.cc/en/main/software

ESP32 Support

Follow the Installation Instructions to add ESP32 support if you re not yet do it:

https://github.com/espressif/arduino-esp32

Arduino_GFX Library

Download latest Arduino_GFX libraries: (press "Clone or Download" -> "Download ZIP")

https://github.com/moononournation/Arduino_GFX

Import libraries in Arduino IDE. (Arduino IDE "Sketch" Menu -> "Include Library" -> "Add .ZIP Library" -> select downloaded ZIP file)

Compile & Upload

1. Open Arduino IDE
2. Open ESP32PhotoFrame sample code ("File" -> "Example" -> "GFX Library for Arduino" -> "WiFiPhotoFrame")
3. Fill your WiFi AP settings into SSID_NAME and SSID_PASSWORD
4. Replace your server hostname or IP and port in HTTP_HOST and HTTP_PORT
5. Press Arduino IDE "Upload" button
6. If you find the orientation not correct, change the "rotation" value (0-3) in new class code

Step 10: Enjoy Photo!

It's time to put the IoT photo frame on your desktop and enjoy!

Step 11: What's Next?

• Add your own instant information
• Fine tune the source photo size for better face detection accuracy
• Automatic task to put latest photos to the server photo folder
• Take more photos ;>