Creating video conference application with GStreamer

GStreamer is a library for constructing graphs of media-handling components. The applications it supports range from simple Ogg/Vorbis playback, audio/video streaming to complex audio (mixing) and video (non-linear editing) processing. GStreamer is released under the LGPL, so it can be used in commercial applications.

The power of GStreamer is in its module infrastructure - you're working with certain "blocks", combine it with "pipes" and got results without any coding (via gst-launch utility). Of course, after playing with gst-launch you can write an application in any language you prefer (GStreamer supports various bindings and due to fact that GStreamer is written in C, I do not see any problems to bind it with an application that is developed in any serious programming language).

I do not want to explain GStream basics, but move forward to the blog topic - creating videoconference application. If you want certain tutorial, please take a look to the Python GStreamer Tutorial by Jens Persson.

Actually, the videoconference application is already written. Of course, it doesn't have any UI, but instead it have all required functionality. You can find it at gstreamer gst-plugins-good RTP examples: server-v4l2-H264-alsasrc-PCMA.sh and client-H264-PCMA.sh. Please notice that the server script broadcast traffic via RTP-protocol, so such conference will not work via plain Internet connection, but only in LAN/VLAN. These scripts require x264enc element, that is provided by gstreamer0.10-plugins-bad-multiverse package in Ubuntu:
$ apt-get install gstreamer0.10-plugins-bad-multiverse
To play with these scripts locally it will be nice to clone the required Git repo and works in your own branches:
$ git clone git://anongit.freedesktop.org/git/gstreamer/gst-plugins-good
Create your own branch and modify sources as you want. For example:
$ cd gst-plugins-good
$ git checkout -b "tests"
You can apply a patch below to make these scripts works locally without any webcams:
diff --git a/tests/examples/rtp/client-H264-PCMA.sh b/tests/examples/rtp/client-H264-PCMA.sh
index ef5e2df..c6aea6d 100755
--- a/tests/examples/rtp/client-H264-PCMA.sh
+++ b/tests/examples/rtp/client-H264-PCMA.sh
@@ -47,7 +47,7 @@ LATENCY=200
 VIDEO_CAPS="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264"
 AUDIO_CAPS="application/x-rtp,media=(string)audio,clock-rate=(int)8000,encoding-name=(string)PCMA"
 
-gst-launch -v gstrtpbin name=rtpbin latency=$LATENCY                                  \
+gst-launch-0.10 -v gstrtpbin name=rtpbin latency=$LATENCY                                  \
      udpsrc caps=$VIDEO_CAPS port=5000 ! rtpbin.recv_rtp_sink_0                       \
        rtpbin. ! rtph264depay ! ffdec_h264 ! xvimagesink                              \
      udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0                                       \
diff --git a/tests/examples/rtp/server-v4l2-H264-alsasrc-PCMA.sh b/tests/examples/rtp/server-v4l2-H264-alsasrc-PCMA.sh
index d42b512..7fe3259 100755
--- a/tests/examples/rtp/server-v4l2-H264-alsasrc-PCMA.sh
+++ b/tests/examples/rtp/server-v4l2-H264-alsasrc-PCMA.sh
@@ -47,8 +47,8 @@ VOFFSET=0
 AOFFSET=0
 
 # H264 encode from the source
-VELEM="v4l2src"
-#VELEM="videotestsrc is-live=1"
+#VELEM="v4l2src"
+VELEM="videotestsrc is-live=1"
 VCAPS="video/x-raw-yuv,width=352,height=288,framerate=15/1"
 VSOURCE="$VELEM ! $VCAPS ! queue ! videorate ! ffmpegcolorspace"
 VENC="x264enc byte-stream=true bitrate=300 ! rtph264pay"
@@ -58,8 +58,8 @@ VRTCPSINK="udpsink port=5001 host=$DEST sync=false async=false name=vrtcpsink"
 VRTCPSRC="udpsrc port=5005 name=vrtpsrc"
 
 # PCMA encode from the source
-AELEM="alsasrc"
-#AELEM="audiotestsrc is-live=1"
+#AELEM="alsasrc"
+AELEM="audiotestsrc is-live=1"
 ASOURCE="$AELEM ! queue ! audioresample ! audioconvert"
 AENC="alawenc ! rtppcmapay"
 
@@ -67,7 +67,7 @@ ARTPSINK="udpsink port=5002 host=$DEST ts-offset=$AOFFSET name=artpsink"
 ARTCPSINK="udpsink port=5003 host=$DEST sync=false async=false name=artcpsink"
 ARTCPSRC="udpsrc port=5007 name=artpsrc"
 
-gst-launch -v gstrtpbin name=rtpbin \
+gst-launch-0.10 -v gstrtpbin name=rtpbin \
     $VSOURCE ! $VENC ! rtpbin.send_rtp_sink_0                                             \
         rtpbin.send_rtp_src_0 ! $VRTPSINK                                                 \
         rtpbin.send_rtcp_src_0 ! $VRTCPSINK                                               \
As you can see, I just changed AELEM and VELEM in the server script, and fixed a gst-launch utility name. After that you can start a server:
$ cd tests/examples/rtp
$ ./server-v4l2-H264-alsasrc-PCMA.sh 
And in another terminal - a client:
$ ./client-H264-PCMA.sh
That's all! You will see a picture and hear a certain noise:
Now you can change VELEM and AELEM to required sources (including your own) and embed GStream graph to your application. Good luck!

UPDATE: In new versions of Ubuntu the x264enc gstreamer element has been moved to another package: gstreamer0.10-plugins-ugly-multiverse

Comments

  1. Hi, I have tested the scripts and they work pretty well, thanks. I have just a doubt. I get a delay of about 2-3 seconds running the server and the client in the same computer. Can this delay be reduced?

    Thank you in advance

    ReplyDelete
  2. Actually, I don't have any noticeable delays running the server and the client in the same computer (tested with Ubuntu 10.04, GStreamer 0.10.14). But I would like to suggest changing LATENCY constant in client-H264-PCMA.sh (set it to 0, or some minimal value).

    ReplyDelete

Post a Comment

Popular posts from this blog

Web application framework comparison by memory consumption

Trac Ticket Workflow

Shellcode detection using libemu