GSoC 2021: Final report for multistream support in the GStreamer element

../_images/GSoC-icon-192.png

This is Google Summer of Code 2021 project under libcamera aimed at achieving:

  1. Add support to libcamera’s GStreamer element to support multistreaming.
  2. Add unit test to test GStreamer element functionality in libcamera.

Project Details

Brief overview

The GStreamer libcamera element allows libcamera to be used in GStreamer pipelines. libcamera supports simultaneous streaming, for example having one lower quality stream for preview, while another higher quality stream is used for recording. This project extends support for the GStreamer libcamera element to support this multistreaming.

Usage

Multistream is supported on IPU3, RockChip RK3399, Raspberry Pi 3 and 4 devices camera pipeline. This will only work on those devices.

  1. Install libcamera and GStreamer using this guide: https://libcamera.org/getting-started.html

  2. If the GStreamer element isn’t installed system wide, set the environment variable specifying the location:

    export GST_PLUGIN_PATH=$BUILDPATH/build/src/gstreamer
    
    # here $BUILDPATH points to the build directory.
    
  3. Install gst-plugins-bad as follows, it is required for autovideosink

    sudo apt install gstreamer1.0-plugins-bad
    
  4. Run the following command on the terminal

    gst-launch-1.0 libcamerasrc name=src src.src ! queue ! videoconvert ! autovideosink src.src_0 ! queue ! videoconvert ! autovideosink
    

If it runs correctly, you should see such two windows with camera output as follows:

../_images/multistream-pass.png

Examples

This functionality can be done programmatically, the example codes are here

  1. Single Stream
  2. Multi Stream
  3. Using Device Provider

Notes

  • Use the following command to verify working of multistream on Raspberry Pi 4B+ using Raspberry Pi Cam v1

    gst-launch-1.0 libcamerasrc camera-name="/base/soc/i2c0mux/i2c@1/ov5647@36" name=src src.src ! queue ! videoconvert ! autovideosink src.src_0 ! queue ! videoconvert ! autovideosink`
    
  • Use the following command to compile gst code

    gcc basic-tutorial-3.c -o basic-tutorial-3 `pkg-config --cflags --libs gstreamer-1.0`
    
  • Use the following to view camera using libcamerasrc (replace “Camera 1” with the actual name of the camera)

    gst-launch-1.0 libcamerasrc camera-name="Camera 1" ! videoconvert ! autovideosink
    
  • Use the following to export GStreamer libcamera element, execute this in libcamera root path

    export GST_PLUGIN_PATH=$(pwd)/build/src/gstreamer
    

Conclusion

All goals that were originally proposed have been completed. I will continue to support this project in the future, take care of any bugs, issues and will continue implementing additional features whenever required.

It has been a really great experience for me. I have learned so much in these 10 weeks, which will surely help me achieve my future goals in life. I would like to thank libcamera and Google for giving me such an amazing opportunity. Furthermore, I would also like to thank my mentors Paul and Nicolas for helping me out and supporting me throughout the project, also the members of libcamera community, namely Laurent, Kieran, Umang and Jacopo.

On an ending note, if my words or actions hurt anyone, I am extremely sorry !