Creating video conference application with GStreamer

GStreamer is a library for constructing graphs of media-handling components. The applications it supports range from simple Ogg/Vorbis playback, audio/video streaming to complex audio (mixing) and video (non-linear editing) processing. GStreamer is released under the LGPL, so it can be used in commercial applications.

The power of GStreamer is in its module infrastructure - you're working with certain "blocks", combine it with "pipes" and got results without any coding (via gst-launch utility). Of course, after playing with gst-launch you can write an application in any language you prefer (GStreamer supports various bindings and due to fact that GStreamer is written in C, I do not see any problems to bind it with an application that is developed in any serious programming language).

I do not want to explain GStream basics, but move forward to the blog topic - creating videoconference application. If you want certain tutorial, please take a look to the Python GStreamer Tutorial by Jens Persson.

Actually, the videoconference application is already written. Of course, it doesn't have any UI, but instead it have all required functionality. You can find it at gstreamer gst-plugins-good RTP examples: server-v4l2-H264-alsasrc-PCMA.sh and client-H264-PCMA.sh. Please notice that the server script broadcast traffic via RTP-protocol, so such conference will not work via plain Internet connection, but only in LAN/VLAN. These scripts require x264enc element, that is provided by gstreamer0.10-plugins-bad-multiverse package in Ubuntu:

$ apt-get install gstreamer0.10-plugins-bad-multiverse

To play with these scripts locally it will be nice to clone the required Git repo and works in your own branches:

$ git clone git://anongit.freedesktop.org/git/gstreamer/gst-plugins-good

Create your own branch and modify sources as you want. For example:

$ cd gst-plugins-good
$ git checkout -b "tests"

You can apply a patch below to make these scripts works locally without any webcams:

diff --git a/tests/examples/rtp/client-H264-PCMA.sh b/tests/examples/rtp/client-H264-PCMA.sh
index ef5e2df..c6aea6d 100755
--- a/tests/examples/rtp/client-H264-PCMA.sh
+++ b/tests/examples/rtp/client-H264-PCMA.sh
@@ -47,7 +47,7 @@ LATENCY=200
 VIDEO_CAPS="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264"
 AUDIO_CAPS="application/x-rtp,media=(string)audio,clock-rate=(int)8000,encoding-name=(string)PCMA"
 
-gst-launch -v gstrtpbin name=rtpbin latency=$LATENCY                                  \
+gst-launch-0.10 -v gstrtpbin name=rtpbin latency=$LATENCY                                  \
      udpsrc caps=$VIDEO_CAPS port=5000 ! rtpbin.recv_rtp_sink_0                       \
        rtpbin. ! rtph264depay ! ffdec_h264 ! xvimagesink                              \
      udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0                                       \
diff --git a/tests/examples/rtp/server-v4l2-H264-alsasrc-PCMA.sh b/tests/examples/rtp/server-v4l2-H264-alsasrc-PCMA.sh
index d42b512..7fe3259 100755
--- a/tests/examples/rtp/server-v4l2-H264-alsasrc-PCMA.sh
+++ b/tests/examples/rtp/server-v4l2-H264-alsasrc-PCMA.sh
@@ -47,8 +47,8 @@ VOFFSET=0
 AOFFSET=0
 
 # H264 encode from the source
-VELEM="v4l2src"
-#VELEM="videotestsrc is-live=1"
+#VELEM="v4l2src"
+VELEM="videotestsrc is-live=1"
 VCAPS="video/x-raw-yuv,width=352,height=288,framerate=15/1"
 VSOURCE="$VELEM ! $VCAPS ! queue ! videorate ! ffmpegcolorspace"
 VENC="x264enc byte-stream=true bitrate=300 ! rtph264pay"
@@ -58,8 +58,8 @@ VRTCPSINK="udpsink port=5001 host=$DEST sync=false async=false name=vrtcpsink"
 VRTCPSRC="udpsrc port=5005 name=vrtpsrc"
 
 # PCMA encode from the source
-AELEM="alsasrc"
-#AELEM="audiotestsrc is-live=1"
+#AELEM="alsasrc"
+AELEM="audiotestsrc is-live=1"
 ASOURCE="$AELEM ! queue ! audioresample ! audioconvert"
 AENC="alawenc ! rtppcmapay"
 
@@ -67,7 +67,7 @@ ARTPSINK="udpsink port=5002 host=$DEST ts-offset=$AOFFSET name=artpsink"
 ARTCPSINK="udpsink port=5003 host=$DEST sync=false async=false name=artcpsink"
 ARTCPSRC="udpsrc port=5007 name=artpsrc"
 
-gst-launch -v gstrtpbin name=rtpbin \
+gst-launch-0.10 -v gstrtpbin name=rtpbin \
     $VSOURCE ! $VENC ! rtpbin.send_rtp_sink_0                                             \
         rtpbin.send_rtp_src_0 ! $VRTPSINK                                                 \
         rtpbin.send_rtcp_src_0 ! $VRTCPSINK                                               \

As you can see, I just changed AELEM and VELEM in the server script, and fixed a gst-launch utility name. After that you can start a server:

$ cd tests/examples/rtp
$ ./server-v4l2-H264-alsasrc-PCMA.sh 

And in another terminal - a client:

$ ./client-H264-PCMA.sh

That's all! You will see a picture and hear a certain noise:
Now you can change VELEM and AELEM to required sources (including your own) and embed GStream graph to your application. Good luck!

UPDATE: In new versions of Ubuntu the x264enc gstreamer element has been moved to another package: gstreamer0.10-plugins-ugly-multiverse