I have been following the Jasper project for a while now. I installed it a couple months ago on my raspberry pi, and got some rudimentary features working. Now that Pi is in full service running a home automation system and I have received my next single board computer, the Beaglebone black . Having nothing better to do with it, I decided to see how far Jasper has come and try to get it running on the black. below are my efforts.
Here is my setup;
A Beaglebone black rev C running the 2014-05-14 SD card Debian image. My sound card is a Logitech usb headset, but any USB soundcard should work. I will eventually be getting this one and will report back once it arrives.
I started with the default SD card Debian image for the BBB available here. Although you shouldn’t have any problem installing it directly on rev C boards with 4 gb of emmc. – This has not been tested.
As for accessing the BBB command line, I will be using an ssh session as root through putty. The cloud 9 ide should work, but is not tested at this point.
Much of the documentation in this page is ripped straight from here and has has small changes that enable it to run on the BBB. (To a point, not everything is working yet)
Here goes nothing.
Boot up your BBB and ssh into it.
Run the following commands to update BBB
sudo apt-get update sudo apt-get upgrade -y cd /opt/scripts/tools/ git pull sudo ./update_kernel.sh sudo reboot
You will eventually have to reboot for the update to complete. If anything fails, such as you getting an “out of space” message, you may need to pop the sd card into a Ubuntu machine and expand the filesystem with gparted.
Install some useful tools.
sudo apt-get install vim git-core espeak python-dev python-pip bison libasound2-dev libportaudio-dev python-pyaudio --yes
This is where some things differ between the Rpi and BBB. The Rpi has an audio out port, the BBB does not. (The HDMI does, but I doubt that you’ll be using that) That means that we have to disable the hdmi and make our usb card the default device (card 0)
To disable the hdmi sound card, plug the BBB into your computer with a usb cable. It should show up as a regular flash drive. Find the file called;
Open up the file in your favorite text editor (I use Notepad++) and find the section saying
##Disable HDMI #cape_disable=capemgr.disable_partno=BB-BONELT-HDMI,BB-BONELT-HDMIN
Remove the # from the start of the second line. The section should now look like
##Disable HDMI cape_disable=capemgr.disable_partno=BB-BONELT-HDMI,BB-BONELT-HDMIN
That’s it! Save the file and reboot the BBB.
Once your BBB reboots, plug in your USB sound card and run this command;
You should get a response something like
**** List of PLAYBACK Hardware Devices **** card 1: Headset [Logitech USB Headset], device 0: USB Audio [USB Audio] Subdevices: 1/1 Subdevice #0: subdevice #0
We need to allow the USB sound card to become card 0.
(configuring the sound card probably does not need to be done as we will need to switch out the hw variable in the aplay commands jasper uses shortly anyways. I am just documenting what I have done in my efforts so far.)
Let’s open up an ALSA configuration file in vim:
sudo vim /etc/modprobe.d/alsa-base.conf
Comment out the following line:
options snd-usb-audio index=-2
It should now look like this:
#options snd-usb-audio index=-2
The raspi commands are slightly different from this, instead of commenting out the like, they changed -2 to 0. When I tried this and reloaded alsa (next step) my sound card dissapeared from the # aplay -l command, but could be verified that it was still connected via # lsusb.
Back in the shell, run:
sudo alsa force-reload aplay -l
You should now see something like
root@beaglebone:~/jasper# sudo alsa force-reload Unloading ALSA sound driver modules: snd-usb-audio snd-hwdep snd-usbmidi-lib. Loading ALSA sound driver modules: snd-usb-audio snd-hwdep snd-usbmidi-lib. root@beaglebone:~/jasper# aplay -l **** List of PLAYBACK Hardware Devices **** card 0: Headset [Logitech USB Headset], device 0: USB Audio [USB Audio] Subdevices: 1/1 Subdevice #0: subdevice #0
Next, test that recording works (you may need to restart your BBB) by recording some audio with the following command:
Make sure you have speakers or headphones connected to the audio jack of your BBB. You can play back the recorded file:
This is slightly different then the command in the raspi setup docs, which is
aplay -D hw:1,0 temp.wav
This will cause a problem later as the sound-card HW id’s are hardcoded into Jasper (Developers- please put mic and speaker id’s into a config file!!) We will have to go into the jasper scripts and switch the id’s out manually later.
If all of the steps above succeed, continue on.
Add the following line to the end of ~/.bash_profile (you may need to run
touch ~/.bash_profile if the file doesn’t exist already):
export LD_LIBRARY_PATH="/usr/local/lib" source .bashrc
And this to your ~/.bashrc or ~/.bash_profile:
LD_LIBRARY_PATH="/usr/local/lib" export LD_LIBRARY_PATH PATH=$PATH:/usr/local/lib/ export PATH
With that, we’re ready to install the core software that powers Jasper.
We will be installing jasper in the home folder. Putting it in a folder has not been successfully tested.
Jasper uses Pocketsphinx for voice recognition. Let’s download and unzip the sphinxbase and pocketsphinx packages:
wget http://downloads.sourceforge.net/project/cmusphinx/sphinxbase/0.8/sphinxbase-0.8.tar.gz wget http://downloads.sourceforge.net/project/cmusphinx/pocketsphinx/0.8/pocketsphinx-0.8.tar.gz tar -zxvf sphinxbase-0.8.tar.gz tar -zxvf pocketsphinx-0.8.tar.gz
Now we build and install sphinxbase:
cd ~/sphinxbase-0.8/ ./configure --enable-fixed make sudo make install
cd ~/pocketsphinx-0.8/ ./configure make sudo make install
Once the installations are complete, restart your BBB.
Note that some of these installation steps take more time to complete than previous steps.
Begin by installing some dependencies:
sudo apt-get install subversion autoconf libtool automake gfortran g++ --yes
Next, move into your home (or Jasper) directory to check out and install CMUCLMTK:
svn co https://svn.code.sf.net/p/cmusphinx/code/trunk/cmuclmtk/ cd cmuclmtk/ sudo ./autogen.sh && sudo make && sudo make install cd ..
Then, when you’ve left the CMUCLTK directory, download the following libraries:
wget http://distfiles.macports.org/openfst/openfst-1.3.3.tar.gz wget https://mitlm.googlecode.com/files/mitlm-0.4.1.tar.gz wget https://phonetisaurus.googlecode.com/files/phonetisaurus-0.7.8.tgz wget http://phonetisaurus.googlecode.com/files/g014b2b.tgz
Untar the downloads:
tar -xvf openfst-1.3.3.tar.gz tar -xvf phonetisaurus-0.7.8.tgz tar -xvf mitlm-0.4.1.tar.gz tar -xvf g014b2b.tgz
Build OpenFST: !!! Do not attempt this step, It will take a really long time and fail !!!
cd openfst-1.3.3/ sudo ./configure --enable-compact-fsts --enable-const-fsts --enable-far --enable-lookahead-fsts --enable-pdt sudo make install # come back after a really long time
Create and mount a swap file
These commands will be added later - once jasper gets to the point where it is processing audio on the BBB. if you run into this tutorial before it is updated, just follow the swap file guide linked above.
After these compiles are done, the swap-file is no longer needed. It can be removed after everything is built. Otherwise it may wear down your flash storage.
I’m not sure if this next part is actually needed, as the swap file was the last problem I solved before openfst compiled, (if someone could confirm this that would be great, otherwise once everything is working I’ll install jasper following all instructions in this guide.) but I’m documenting everything anyway with the hope that this will be extremely easy to turn into a guide for jasper on the BBB.
Change the gcc compiler
sudo apt-get install gcc-4.4 g++-4.4 cpp-4.4 sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.3 10 sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.4 20 sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.3 10 sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.4 20 sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc 30 sudo update-alternatives --set cc /usr/bin/gcc sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++ 30 sudo update-alternatives --set c++ /usr/bin/g++
Now that we have the right compiler and enough memory, go ahead and install openfst. I am using screen here as well as I don’t want an accidental ssh closure to stop my build. It will take a couple of hours.
cd openfst-1.3.3/ cpufreq-set -g performance # Increase cpu frequency to 1000mhz for faster compiling. Usually set at 300 - 600mhz screen sudo ./configure --enable-compact-fsts --enable-const-fsts --enable-far --enable-lookahead-fsts --enable-pdt sudo make install # come back after a really long time
OpenFST should compile correctly after this. If you have to leave, disconnect from the screen session using:
CTRL + A D
And reconnect once you log in again using
Once the compile is finished, move onto the next steps.
cd ~ nano /etc/apt/sources.list
deb http://ftp.debian.org/debian experimental main contrib non-free
to the bottom of the file.
CTRL + X , then
enter to save and close. we have now added experimental packages to our sources.
install the experimental package
apt-get update apt-get -t experimental install m2m-aligner
cd .. cd mitlm-0.4.1/ sudo ./configure sudo make install
cd .. cd phonetisaurus-0.7.8/ cd src sudo make
Move some of the compiled files:
sudo cp ~/jasper/phonetisaurus-0.7.8/phonetisaurus-g2p /usr/local/bin/phonetisaurus-g2p
Build Phonetisaurus model: (If you get dictionary errors, reboot the black and rebuild this)
cd g014b2b/ ./compile-fst.sh
Finally, rename the following folder for convenience:
mv ~/g014b2b ~/phonetisaurus
At this point, we’ve installed Jasper and all the necessary software to run it. Before we start playing around, though, we need to configure Jasper and provide it with some basic information.
You can follow the normal jasper instructions from here. things are somewhat working (jasper starts) but there seems to be some bitrate and soundcard issues still.
During my junior year at the EGL Academy we were approached about constructing some audio stations for the Marathon County Historical Society main museum. Being an engineering school with students learning through project based learning, we accepted and immediately put together a team to design, build and install the stations at the museum. I was part of this team, Over A roughly three month planning process (along with regular school, meeting roughly once every two weeks) I was eventually promoted to leader and head of the project. Fulfilling the roles of organisation, installation, electronics construction (we used a custom electronics setup), and the complete design and modification of the antique radio that was to become the fourth audio station. Working with three other students – Trenton, Andrew, (construction) and Thomas (audio editing) – We built the three pedestal based audio stations, completing the wood structure just before the end of the school year. While Thomas took over 15 hours of audio and compressed it into 20 2-5 minute clips for the audio stations. Completing the files and sending them to me just after school got out. at this point we had gone to many meetings, picked up the radio, gutted it (but nothing else on that audio station), built the wood structure for the three four button pedestals (the radio was going to have eight), and ordered nearly all electronics and started building prototypes. this is the point we were at leading up to and in the pictures below.
This is the Radio just after gutting, old tube electronics and face-plate removed for modification.
recieved a wrong switch, (clear one) and contacting adafruit.com customer service.
Read the service request here.
Next Pictures are of Andrew, Trenton and I constructing the pedestals.
Planing rough Lumber for stands (Pine, if I remember correctly)
Gluing boxes together
After this point we had made and blued all the pieces together, so Trenton and Andrew left for the night…. Never to be seen again….(Actually they decided their part in the project was over, as school ended in about two days)
The next step in the process was to stain the boxes, finish assembly (screw everything together that wasn’t glued), and laser gut the glare proof acrylic for the switches. These are the next pictures.
After this I installed the buttons and wired the electronics up and tested them. They worked flawlessly with the code I had developed over the last month. (Get it here) Unfortunately, I do not have a picture of this step in the process. But soon after I meet up With Russ & Linda (The Museum Curators) and we installed the first working pedestal.
(You cant see it here, but the buttons are softly pulsing and turn solid when a button is pressed to play an audio file.)
After the first pedestal, (Of which Russ & Linda were ecstatic to have turned out so well) We installed the rest of the three, 4 button stations.
Work themed station #1 installed
Work themed station #2 installed
The next part of the project was to modify the radio into an audio station. this was much different from the others and presented its own unique set of problems because
- We wanted to keep the original radio experience by having a speaker system play the files out-loud, but because the museum is so small and has cement floors, the sound echos throughout the whole museum.
- This station uses eight buttons instead of four, necessitating a brand new program and hardware on the controller.
- (partial fix to # 1) Where do you put the buttons and headphones so everyone, from people in wheelchairs to 7″ tall people, can reach the buttons and have enough cable length to bring the headsets up to their ears?
After much discussion with Russ, the answers to these questions turned out to be:
- Install headphones for everyday usage, but install a button to switch between the headphones and speakers for group visits that go to one station at a time.
- Rewrite code and add extra inputs and outputs to hardware.
- Place the radio as close as possible to the edge and place the handsets on the front of the radio.
Build progress is below.
Buttons installed in faceplate
Radio installed at museum
With all the hardware installed the next round was checking the functionality of the stations and the accuracy of the audio files. And there was a problem. About half of the audio files were missing clips or ended one or two words early. Thomas was on vacation and unable to fix the files, he also hadn’t saved the edit files after he was finished. leaving me to learn the audio editing program (Audacity) and completely reedit the files correctly for the audio station. This put me over the 90 hour mark for my junior capstone project. You can download all files (audio, programming, instruction manual, pictures) at the end of this article.
After We got the audio files sorted out the only thing I had left to do was to Write the instruction manual for the custom electronics. This was decided to be done so others can use these electronics long in the future, as well as fix them if everything is wrong. This put me at a grand total of 93 logged hours (Probably much more in reality) of group leadership, working with businesses, and volunteer work.
During this whole project I gained many new skills and experiences. working with a museum system gave me a behind the scenes look at how museums are built an operate. As well as a few great new contacts in Wausau. This project also helped me get more comfortable in a leadership role. Teaching me what people expect out of me, how to predict and meet deadlines, and how to organize meetups, working with everyone’s schedule to make it work for them. I also learned many things in my hands on construction of the audio stations, such as how to optimize code in Arduino and what to look for in making the audio stations easily accessible to everyone.
All in all, I think this project made a difference in me and benefited the community, and I would definitely go through this experience again.
- Pictures: http://www.flickr.com/photos/76027390@N05/sets/72157635785424065/
- Arduino program: 4/8 Button audio display driver.zip
- Manual: Audio Station Electronics Setup and Usage Manual
- Audio files: MCHS audio station audio.zip
Iv’e decided to start this blog for my projects and explorations in engineering after seeing and enjoying someone that has done very much the same thing. This blog will try to track my progress in the many projects that I do throughout the summer. Then when school is in session, the things I do in the project based charter school I go to. I will be backlogging some of the projects that came about before this blog (Most notably my CNC Machine, Tesla coil, and 3D Printer. Listed in succession to when I made them) and will post updates on things as needed.