VR Moving Binocular ESP32-cams

Video conferencing can be boring, right? Better to see everything in 3D. Stereoscopic pictures are fascinating, so I decided to catch an image for each eye and send this to VR glasses. This tutorial shows how to build a moving VR camera with simple tools.

It is for makers with experience in programming and uploading of Arduino and ESP32.

Step 1: Material

Eyes:

• 2x ESP32-cam

Head:

• NodeMCU 8266 Amica
• 3 Servos
• pan & tilt kit
• USB-Header

VR display:

• glasses (~20 €)
• ESP8266-01
• MPU 6050
• LiPo battery

Additional:

• smartphone
• soldering station + wire
• pins, sockets
• circuit boards
• USB cables, power supplies/powerbanks

Codes:

• ESP32-cam1
• ESP32-cam2
• NodeMCU Amica
• ESP8266-01

Step 2: Camera Eyes

First you load the scripts on the two ESP32-cams. How, you can see here e.g :

https://www.instructables.com/ESP32-CAM-WEB-Server...

The ESP cameras are mounted with sockets on a PCB with a common 5V power supply. If you want, you can add pin connectors or a USB socket. Important is the distance of the lenses of 6,5 cm, which is about the distance of the eyes of an adult. How the programming of an ESP-Cam works, you can read e.g. here: https://www.instructables.com/Getting-Started-Wit...

Camera1 - For uploading the following files should be in one folder: ESP-cam1_AP_webserver.ino, camera_pins.h, home_wifi_multi.h, VR-Eyes.h.

For Camera 2 you need: ESP-cam2.ino, camera_pins.h, home_wifi_multi.h in one folder.

"OV2640.h" and "OV2640.cpp" must be in a subdirectory "src" for each camera.

Camera 1 (left eye) acts as access point and server, it also provides the web page where both camera streams are embedded. Now connect your smartphone (or your PC) with the WIFI "ESP32 AP"of camera 1 and use your browser to go to the address 192.168.4.1. Voila!

Step 3: VR Glasses

Next, we want to see our two images stereoscopically. To do this, I got myself a simple pair of VR goggles with two lenses.

markso9 has a nice tutorial on how to build it yourself out of cardboard:
https://www.instructables.com/Make-Your-Own-Virtual-Reality-Glasses/

If you log into "ESP32 AP" with your cell phone, go to 192.168.4.1 and mount your smartphone on the VR glasses, it's already a nice 3D experience.

I am also considering mounting this previous result on an RC car or drone.

Step 4: Servo Unit

This was not enough for me, I wanted to be able to move the 3D image as well. From a pan&tilt-kit and three servos I built a "mechanical head", with which the cameras can rotate, tilt and tilt sideways. The servos are controlled by a NodeMCU Amica. Alternatively, any other ESP8266 or ESP32 with enough slots could be used. I mounted the ESP8266 again with sockets on a PCB.

I have separated the power supply of ESP8266 and servos/cameras and use a dual power device.

Connect the servo data lines to pins 6, 7 and 8 of the ESP8266, volts and ground to the separate power supply. Do not forget to connect the ground of the servos to the ground of the NodeMcu.

As our camera1 already provides a WIFI, we log in there with the servo unit and later also with the head tracker.

Step 5: Head Tracker

Head movements are now to be transmitted from the VR goggles to the servo unit. The movements are captured by a MPU6050 and the values are sent via an ESP8266-01. X- and Y- values (tilt and sideways tilt) can be captured very nicely with the gyroscope function, the values converted a bit and sent to the servo unit. Unfortunately, the Z-rotation is only captured relatively, after the rotation the value goes right back to zero. Therefore I took only the change of the variable at Z and added it to the current servo position.

Any other ESP can be used as a transmitter, I tried it with an ESP8285-M3, but could not get it to work. You can find a good ESP-01-tutorial here:

https://www.taydaelectronics.com/datasheets/files/ESP-01S.pdf

The tracker is mounted together with the battery on the VR goggles and sends signals only to 192.168.2.13 (servo unit) in our little WIFI network. Change this address in the code if necessary.

Step 6: Conclusion

Now you can put the servo unit in the next room and make your observations in 3D.

Future improvements:
- more stable construction of the servo unit

- let the devices operate in the home network to increase the range of coverage

- make the transmissions of the streams and VR-values via internet

Have fun and let me know!

Step 7:

Step 8: Update

Some users have complained that the connection is too slow. No wonder, ESP32-Cam1 has to be access point, web server and camera at the same time. We were thinking of using another ESP32 and distributing the tasks:

ESP32: AP and webserver

Cam1  + Cam2. only responsible for the pictures

I have not yet tested the cooperation of the three ESPs, I am just putting them here for discussion.