Home > ARM, Development boards, Embedded Linux, Guides, Raspberry Pi, TKJ Electronics > How to stream video and audio from a Raspberry Pi with no latency

How to stream video and audio from a Raspberry Pi with no latency

I have finally finished my last exams, so now I have more time to focus on some of my own projects. It has been a while since our Kickstarter campaign was successfully funded, but we are still working on making the experience better for the final users.

After the campaign ended we sent out a survey to all our backers with several questions about there address, profession and so on, but we also asked them if they had any suggestions for improvements or extra features they would like to see added to the Balanduino. A lot of people asked if we could enable wireless streaming for it.
I was personally very excited about that since I have been playing with the thought for quite a while, so when the official camera module for the Raspberry Pi became available I bought it straight away.

Raspberry Pi camera module

If you do not have much experience with the Raspberry Pi I recommend reading Thomas’s blog post he wrote a while ago: http://blog.tkjelectronics.dk/2012/09/raspberry-pi-gpio-control/ and also to viset the official homepage: http://www.raspberrypi.org/.

I really have not used by Raspberry Pi that much – it has been laying on my desk for quite some time, as I did not have any real use for it until now. I have tried some different approaches, but ended up using a something called gstreamer 1.0. Note that there is currently no official binaries for Windows, so you will have to compile them yourself, but it works great on both Mac OS X and Linux.

I am running Arch Linux on my Raspberry Pi since I do not need the desktop environment that comes with the official Raspbian “wheezy” image, but it should not make much of a difference.

I will not go into much detail on how to install gstreamer 1.0 on your computer, but instead I will refer to the page I originally got the commands from: http://pi.gbaman.info/?p=150.

Also check out the original forum post: http://www.raspberrypi.org/phpBB3/viewtopic.php?uid=56951&f=43&t=44987&start=0.

Video

To stream video from my Raspberry Pi I run the following command:

raspivid -t 999999 -w 1080 -h 720 -fps 25 -hf -b 2000000 -o - | \
gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 \
! gdppay ! tcpserversink host=serverIp port=5000

And to receive the video on my Mac I run the following command in the terminal:

gst-launch-1.0 -v tcpclientsrc host=serverIp port=5000 \
! gdpdepay ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false

Be sure to replace serverIp with the ip of your Raspberry Pi. You can get it by running the following command on the Raspberry Pi:

ifconfig

The video quality is really great and there is basically no latency at all.

Audio

To stream audio turned out to be a little harder to get working. There is no audio input on the Raspberry Pi, so I used an old Icemat USB Sound card I had laying around.

First of all type the following command into your Raspberry Pi:

lsusb

It should print out something like this:

Bus 001 Device 002: ID 0424:9512 Standard Microsystems Corp. LAN9500 Ethernet 10/100 Adapter / SMSC9512/9514 Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp. SMSC9512/9514 Fast Ethernet Adapter
Bus 001 Device 004: ID 0d8c:000c C-Media Electronics, Inc. Audio Adapter

Note the last device is my USB sound card.

Btw I recommend using ssh to login to your device. In fact I never use the HDMI output, as I prefer using my laptop. For more information check out this guide by Adafruit: http://learn.adafruit.com/adafruits-raspberry-pi-lesson-6-using-ssh/.

If your audio device is not working properly, then I recommend googling your USB sound card, as somebody else have properly already written a guide on how to get it working.

After that you will need to install the following packages: ‘alsa-utils’, ‘alsa-firmware’, and ‘alsa-plugins’.

Since I run Arch Linux I do this by running:

pacman -S alsa-utils alsa-firmware alsa-plugins

After that reboot your device:

reboot

If you run Raspbian “wheezy”:

shutdown -r now

After that run the following command:

cat /proc/asound/cards

This should print out a list of your sound cards like so:

 0 [ALSA           ]: BRCM bcm2835 ALSbcm2835 ALSA - bcm2835 ALSA
                      bcm2835 ALSA
 1 [Set            ]: USB-Audio - C-Media USB Headphone Set
                      C-Media USB Headphone Set at usb-bcm2708_usb-1.2, full speed

Notice that the name of my device is ‘Set’ – we will use this later.

You can also see the name by running the following command:

asoundconf list

To get even more information run:

aplay -l

Now we want to set this sound card as the default. This is done by the following command:

asoundconf set-default-card cardname

Be sure to replace ‘cardname’ with the name of your sound card. In my case I replace it with ‘Set’ like so:

asoundconf set-default-card Set

Also add the following:

pcm.!default {
    type hw
    card Set
    device 0
}
ctl.!default {
    type hw
    card Set
    device 0
}

To: /etc/asound.conf:

nano /etc/asound.conf

Now reboot once more.

Now it is time to test if it is actually working. Run the following command:

speaker-test -c 2

You should hear noise in your headphones.

Okay now try to run the following command:

gst-launch-1.0 audiotestsrc ! alsasink

You should hear a high pitch tone.

Now for something a bit more interesting. Try to run the following command:

gst-launch-1.0 alsasrc device=plughw:Set ! alsasink

Be sure yo replace ‘Set’ with the name of your sound card.
Now the input from the microphone should be played out the audio output of the USB sound card.

To adjust your setting – like volume, microphone level etc. Use the following command:

alsamixer

And then to store your settings:

alsactl store

To make a 10 second audio recording at a samplerate of 16000 bit/s via the microphone run:

arecord -D plughw:Set -r 16000 test.wav -d 10

To play it back:

aplay -D plughw:Set test.wav

Now to the fun part!

To stream the audio run:

gst-launch-1.0 -v alsasrc device=plughw:Set \
! mulawenc ! rtppcmupay ! udpsink host=clientIp port=5001

Replace the clientIp with your computers ip address. Normally I just set the end to 255, so it can be received on every computer on the network, for instance I set mine to: 10.0.0.255.

To receive the audio on your computer:

gst-launch-1.0 -v udpsrc port=5001 caps="application/x-rtp" \
! queue ! rtppcmudepay ! mulawdec ! audioconvert ! autoaudiosink sync=false

If you experience dropouts you might want to add ‘num-buffers=1000′ just after ‘device=plughw:Set’ at the server side, but in my experience it actually works better by not using it.

In the end I decided to create two scripts. One at the streaming end (the Raspberry Pi) and another and the receiver – my computer. This will allow me to stream the video and audio simultaneously from the Raspberry Pi to my computer.

The streaming script:

#!/bin/bash -e

serverIp=$(ifconfig | grep -E 'inet.[0-9]' | grep -v '127.0.0.1' | awk '{ print $2}')
clientIp=$(echo $serverIp | cut -d '.' -f 1-3).255 # Send to all

gst-launch-1.0 -v alsasrc device=plughw:Set \
! mulawenc ! rtppcmupay ! udpsink host=$clientIp port=5001 &

raspivid -t 999999 -w 1080 -h 720 -fps 25 -hf -b 2000000 -o - | \
gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 \
! gdppay ! tcpserversink host=$serverIp port=5000

kill $!

And the receiving script:

#!/bin/bash -e

serverIp=rpi.local

gst-launch-1.0 -v udpsrc port=5001 caps="application/x-rtp" \
! queue ! rtppcmudepay ! mulawdec ! audioconvert ! autoaudiosink sync=false &

gst-launch-1.0 -v tcpclientsrc host=$serverIp port=5000 \
! gdpdepay ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false

kill $!

Make sure to make the script executable:

chmod +x /path/to/your/script.sh

Note that I have setup Bonjour on the Raspberry Pi so I can simply refer to it as hostname.local – in my case rpi.local. More information can be found at the following site: http://gettingstartedwithraspberrypi.tumblr.com/post/24398167109/file-sharing-with-afp-and-auto-discovery-with-bonjour.

I am very satisfied with the performance. There is almost no latency at all both via a wired or a wireless connection.

Fell free to post any questions or comments below and I will answer as quickly as possible! :)

  1. June 20th, 2014 at 16:50 | #1

    @SB YIM
    No you can not view it in a browser, but you will be able to write a dedicated Android app that. Please take a look at the following site for more information: http://www.gstreamer.com/.

Comment pages
1 2 4081
  1. No trackbacks yet.