Installing Jasper on the Beaglebone Black

https://i0.wp.com/www.voltvision.com/wp-content/uploads/2014/04/BBB1.png

I have been following the Jasper project for a while now. I installed it a couple months ago on my raspberry pi, and got some rudimentary features working. Now that Pi is in full service running a home automation system and I have received my next single board computer, the Beaglebone black . Having nothing better to do with it, I decided to see how far Jasper has come and try to get it running on the black. below are my efforts.

Here is my setup;

A Beaglebone black rev C running the 2014-05-14  SD card Debian image. My sound card is a Logitech usb headset, but any USB soundcard should work. I will eventually be getting this one and will report back once it arrives.

I started with the default SD card Debian image for the BBB available here. Although you shouldn’t have any problem installing it directly on rev C boards with 4 gb of emmc.  – This has not been tested.

As for accessing the BBB command line, I will be using an ssh session as root through putty. The cloud 9 ide should work, but is not tested at this point.

Much of the documentation in this page is ripped straight from here and has has small changes that enable it to run on the BBB. (To a point, not everything is working yet)

Here goes nothing.

Manual Install

Boot up your BBB and ssh into it.

Run the following commands to update BBB

sudo apt-get update
sudo apt-get upgrade -y
cd /opt/scripts/tools/
git pull
sudo ./update_kernel.sh
sudo reboot

You will eventually have to reboot for the update to complete. If anything fails, such as you getting an “out of space” message, you may need to pop the sd card into a Ubuntu machine and expand the filesystem with gparted.

Install some useful tools.

sudo apt-get install vim git-core espeak python-dev python-pip bison libasound2-dev libportaudio-dev python-pyaudio --yes

This is where some things differ between the Rpi and BBB. The Rpi has an audio out port, the BBB does not. (The HDMI does, but I doubt that you’ll be using that) That means that we have to disable the hdmi and make our usb card the default device (card 0)

To disable the hdmi sound card, plug the BBB into your computer with a usb cable. It should show up as a regular flash drive. Find the file called;

uEnv.txt

Open up the file in your favorite text editor (I use Notepad++) and find the section saying

##Disable HDMI
#cape_disable=capemgr.disable_partno=BB-BONELT-HDMI,BB-BONELT-HDMIN

Remove the # from the start of the second line. The section should now look like

##Disable HDMI
cape_disable=capemgr.disable_partno=BB-BONELT-HDMI,BB-BONELT-HDMIN

That’s it! Save the file and reboot the BBB.

Once your BBB reboots, plug in your USB sound card and run this command;


aplay -l

You should get a response something like

**** List of PLAYBACK Hardware Devices ****
card 1: Headset [Logitech USB Headset], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

We need to allow the USB sound card to become card 0.
(configuring the sound card probably does not need to be done as we will need to switch out the hw variable in the aplay commands jasper uses shortly anyways. I am just documenting what I have done in my efforts so far.)

Let’s open up an ALSA configuration file in vim:

sudo vim /etc/modprobe.d/alsa-base.conf

Comment out the following line:

options snd-usb-audio index=-2

It should now look like this:

#options snd-usb-audio index=-2

The raspi commands are slightly different from this, instead of commenting out the like, they changed -2 to 0. When I tried this and reloaded alsa (next step) my sound card dissapeared from the # aplay -l command, but could be verified that it was still connected via # lsusb.

Back in the shell, run:

sudo alsa force-reload
aplay -l

You should now see something like

root@beaglebone:~/jasper# sudo alsa force-reload
Unloading ALSA sound driver modules: snd-usb-audio snd-hwdep snd-usbmidi-lib.
Loading ALSA sound driver modules: snd-usb-audio snd-hwdep snd-usbmidi-lib.
root@beaglebone:~/jasper# aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: Headset [Logitech USB Headset], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

Next, test that recording works (you may need to restart your BBB) by recording some audio with the following command:

arecord temp.wav

Make sure you have speakers or headphones connected to the audio jack of your BBB. You can play back the recorded file:

aplay temp.wav

This is slightly different then the command in the raspi setup docs, which is

aplay -D hw:1,0 temp.wav

This will cause a problem later as the sound-card HW id’s are hardcoded into Jasper (Developers- please put mic and speaker id’s into a config file!!) We will have to go into the jasper scripts and switch the id’s out manually later.

If all of the steps above succeed, continue on.

Add the following line to the end of ~/.bash_profile (you may need to run touch ~/.bash_profile if the file doesn’t exist already):

export LD_LIBRARY_PATH="/usr/local/lib"
source .bashrc

And this to your ~/.bashrc or ~/.bash_profile:

LD_LIBRARY_PATH="/usr/local/lib"
export LD_LIBRARY_PATH
PATH=$PATH:/usr/local/lib/
export PATH

With that, we’re ready to install the core software that powers Jasper.

Install Pocketsphinx

We will be installing jasper in the home folder. Putting it in a folder has not been successfully tested.

cd ~

Jasper uses Pocketsphinx for voice recognition. Let’s download and unzip the sphinxbase and pocketsphinx packages:

wget http://downloads.sourceforge.net/project/cmusphinx/sphinxbase/0.8/sphinxbase-0.8.tar.gz
wget http://downloads.sourceforge.net/project/cmusphinx/pocketsphinx/0.8/pocketsphinx-0.8.tar.gz
tar -zxvf sphinxbase-0.8.tar.gz
tar -zxvf pocketsphinx-0.8.tar.gz

Now we build and install sphinxbase:

cd ~/sphinxbase-0.8/
./configure --enable-fixed
make
sudo make install

And pocketsphinx:

cd ~/pocketsphinx-0.8/
./configure
make
sudo make install

Once the installations are complete, restart your BBB.

Install CMUCLMTK, OpenFST, MIT Language Modeling Toolkit, m2m-aligner, Phonetisaurus

Note that some of these installation steps take more time to complete than previous steps.

Begin by installing some dependencies:

sudo apt-get install subversion autoconf libtool automake gfortran g++ --yes

Next, move into your home (or Jasper) directory to check out and install CMUCLMTK:

svn co https://svn.code.sf.net/p/cmusphinx/code/trunk/cmuclmtk/
cd cmuclmtk/
sudo ./autogen.sh && sudo make && sudo make install
cd ..

Then, when you’ve left the CMUCLTK directory, download the following libraries:

wget http://distfiles.macports.org/openfst/openfst-1.3.3.tar.gz
wget https://mitlm.googlecode.com/files/mitlm-0.4.1.tar.gz
wget https://phonetisaurus.googlecode.com/files/phonetisaurus-0.7.8.tgz
wget http://phonetisaurus.googlecode.com/files/g014b2b.tgz

Untar the downloads:

tar -xvf openfst-1.3.3.tar.gz
tar -xvf phonetisaurus-0.7.8.tgz
tar -xvf mitlm-0.4.1.tar.gz
tar -xvf g014b2b.tgz

Build OpenFST: !!! Do not attempt this step, It will take a really long time and fail !!!

cd openfst-1.3.3/
sudo ./configure --enable-compact-fsts --enable-const-fsts --enable-far --enable-lookahead-fsts --enable-pdt
sudo make install # come back after a really long time

This is where things failed for me. I eventually had to try a combination of adding a swap file and following the instructions in this bug report. Commands follow.

Create and mount a swap file

These commands will be added later - once jasper gets to the point where it is processing audio on the BBB. 
if you run into this tutorial before it is updated, just follow the swap file guide linked above.

After these compiles are done, the swap-file is no longer needed. It can be removed after everything is built.  Otherwise it may wear down your flash storage.

I’m not sure if this next part is actually needed, as the swap file was the last problem I solved before openfst compiled, (if someone could confirm this that would be great, otherwise once everything is working I’ll install jasper following all instructions in this guide.) but I’m documenting everything anyway with the hope that this will be extremely easy to turn into a guide for jasper on the BBB.

Change the gcc compiler

sudo apt-get install gcc-4.4 g++-4.4 cpp-4.4

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.3 10
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.4 20

sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.3 10
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.4 20

sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc 30
sudo update-alternatives --set cc /usr/bin/gcc

sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++ 30
sudo update-alternatives --set c++ /usr/bin/g++

Now that we have the right compiler and enough memory, go ahead and install openfst. I am using screen here as well as I don’t want an accidental ssh closure to stop my build. It will take a couple of hours.

cd openfst-1.3.3/
cpufreq-set -g performance # Increase cpu frequency to 1000mhz for faster compiling. Usually set at 300 - 600mhz
screen
sudo ./configure --enable-compact-fsts --enable-const-fsts --enable-far --enable-lookahead-fsts --enable-pdt
sudo make install # come back after a really long time

OpenFST should compile correctly after this. If you have to leave, disconnect from the screen session using:

CTRL + A 
D

And reconnect once you log in again using

screen -r

Once the compile is finished, move onto the next steps.

Install experimental m2m

cd ~
nano /etc/apt/sources.list

and paste

deb http://ftp.debian.org/debian experimental main contrib non-free

to the bottom of the file.

press CTRL + X , then enter to save and close. we have now added experimental packages to our sources.

install the experimental package

apt-get update
apt-get -t experimental install m2m-aligner

Build MITLMT:

cd ..
cd mitlm-0.4.1/
sudo ./configure
sudo make install

Build Phonetisaurus:

cd ..
cd phonetisaurus-0.7.8/
cd src
sudo make

Move some of the compiled files:

sudo cp ~/jasper/phonetisaurus-0.7.8/phonetisaurus-g2p /usr/local/bin/phonetisaurus-g2p

Build Phonetisaurus model: (If you get dictionary errors, reboot the black and rebuild this)

cd g014b2b/
./compile-fst.sh

Finally, rename the following folder for convenience:

mv ~/g014b2b ~/phonetisaurus

At this point, we’ve installed Jasper and all the necessary software to run it. Before we start playing around, though, we need to configure Jasper and provide it with some basic information.

You can follow the normal jasper instructions from here. things are somewhat working (jasper starts) but there seems to be some bitrate and soundcard issues still.

Advertisements

Marathon County Historical Socieity Audio Stations

During my junior year at the EGL Academy we were approached about constructing some audio stations for the Marathon County Historical Society main museum. Being an engineering school with students learning through project based learning, we accepted and immediately put together a team to design, build and install the stations at the museum. I was part of this team, Over A roughly three month planning process (along with regular school, meeting roughly once every two weeks) I was eventually promoted to leader and head of the project. Fulfilling the roles of organisation, installation, electronics construction (we used a custom electronics setup), and the complete design and modification of the antique radio that was to become the fourth audio station. Working with three other students – Trenton, Andrew, (construction) and Thomas (audio editing) – We built the three pedestal based audio stations, completing the wood structure just before the end of the school year. While Thomas took over 15 hours of audio and compressed it into 20 2-5 minute clips for the audio stations. Completing the files and sending them to me just after school got out. at this point we had gone to many meetings, picked up the radio, gutted it (but nothing else on that audio station), built the wood structure for the three four button pedestals (the radio was going to have eight), and ordered nearly all electronics and started building prototypes. this is the point we were at leading up to and in the pictures below.

mas iphone after europe 390

This is the Radio just after gutting, old tube electronics and face-plate removed for modification.

junior capstone MCHS
Prototyping electronics

junior capstone MCHS

recieved a wrong switch, (clear one) and contacting adafruit.com customer service.

Read the service request here.

Next Pictures are of Andrew, Trenton and I constructing the pedestals.

PICT2392Planing rough Lumber for stands (Pine, if I remember correctly)

PICT2412
Trenton using wood filler on nail holes

PICT2420Pedestal “assembly line”.  From left to right – Me, Andrew, Trenton.

PICT2413
PICT2405

Gluing boxes together

After this point we had made and blued all the pieces together, so Trenton and Andrew left for the night…. Never to be seen again….(Actually they decided their part in the project was over, as school ended in about two days)

The next step in the process was to stain the boxes, finish assembly (screw everything together that wasn’t glued), and laser gut the glare proof acrylic for the switches. These are the next pictures.

junior capstone MCHSMeasuring buttons for hole size.

junior capstone MCHSlaying out Plexiglas for switch and phone positions (not final)

junior capstone MCHSPlexiglas cut and attached to pedestal. Button holes not drilled yet.

After this I installed the buttons and wired the electronics up and tested them. They worked flawlessly with the code I had developed over the last month. (Get it here)  Unfortunately, I do not have a picture of this step in the process. But soon after I meet up With Russ & Linda (The Museum Curators) and we installed the first working pedestal.

junior capstone MCHS

Its alive!

(You cant see it here, but the buttons are softly pulsing and turn solid when a button is pressed to play an audio file.)

After the first pedestal, (Of which Russ & Linda were ecstatic to have turned out so well) We installed the rest of the three, 4 button stations.

junior capstone MCHS

Immigrant themed station installed

junior capstone MCHS

Work themed station #1 installed

junior capstone MCHS

Work themed station #2 installed

The next part of the project was to modify the radio into an audio station. this was much different from the others and presented its own unique set of problems because

  1. We wanted to keep the original radio experience by having a speaker system play the files out-loud, but because the museum is so small and has cement floors, the sound echos throughout the whole museum.
  2. This station uses eight buttons instead of four, necessitating a brand new program and hardware on the controller.
  3. (partial fix to # 1) Where do you put the buttons and headphones so everyone, from people in wheelchairs to 7″ tall people, can reach the buttons and have enough cable length to bring the headsets up to their ears?

After much discussion with Russ, the answers to these questions turned out to be:

  1. Install headphones for everyday usage, but install a button to switch between the headphones and speakers for group visits that go to one station at a time.
  2. Rewrite code and add extra inputs and outputs to hardware.
  3. Place the radio as close as possible to the edge and place the handsets on the front of the radio.

Build progress is below.

junior capstone MCHS
Adding numbers to buttons

junior capstone MCHS
Buttons finished

junior capstone MCHS

Buttons installed in faceplate

junior capstone MCHSWiring electronics.

junior capstone MCHSButtons wired and SD card installed.

junior capstone MCHSInstalling face-plate.

junior capstone MCHSFace-plate installed.
junior capstone MCHS

junior capstone MCHS

Radio installed at museum

With all the hardware installed the next round was checking the functionality of the stations and the accuracy of the audio files. And there was a problem. About half of the audio files were missing clips or ended one or two words early.  Thomas was on vacation and  unable to fix the files, he also hadn’t saved the edit files after he was finished. leaving me to learn the audio editing program (Audacity) and completely reedit the files correctly for the audio station. This put me over the 90 hour mark for my junior capstone project. You can download all files (audio, programming, instruction manual, pictures) at the end of this article.

After We got the audio files sorted out the only thing I had left to do was to Write the instruction manual for the custom electronics. This was decided to be done so others can use these electronics long in the future, as well as fix them if everything is wrong.  This put me at a grand total of 93 logged hours (Probably much more in reality) of group leadership, working with businesses, and volunteer work.

During this whole project I gained many new skills and experiences. working with a museum system gave me a behind the scenes look at how museums are built an operate. As well as a few great new contacts in Wausau. This project also helped me get more comfortable in a leadership role. Teaching me what people expect out of me, how to predict and meet deadlines, and how to organize meetups, working with everyone’s schedule to make it work for them. I also learned many things in my hands on construction of the audio stations, such as how to optimize code in Arduino and what to look for in making the audio stations easily accessible to everyone.

All in all, I think this project made a difference in me and benefited the community, and I would definitely go through this experience again.

Files:

The purpose of this blog

Iv’e decided to start this blog for my projects and explorations in engineering after seeing and enjoying someone that has done very much the same thing.  This blog will try to track my progress in the many projects that I do throughout the summer. Then when school is in session, the things I do in the project based charter school I go to.  I will be backlogging some of the projects that came about before this blog  (Most notably my CNC Machine, Tesla coil, and 3D Printer.  Listed in succession to when I made them)  and will post updates on things as needed.

– Marshall