14.3.1 Offline sound source localization

14.3.1.1 Execution

Run the offline localization in Localization directory. This script displays the sound source localization result of three 7ch multichannel audio files: a sound from 0 deg (microcone_0deg.wav), a sound from 90 deg (microcone_90deg.wav), and a sound moving around the microcone (microcone_moving.wav). You can see them in data directory.

You can run the script by

> ./demo.sh offline

Then, you will see the output like Figure 14.6 in the terminal, and a graph of sound source localization.

UINodeRepository::Scan()
Scanning def /usr/lib/flowdesigner/toolbox
done loading def files
loading XML document from memory
done!
Building network  :MAIN
TF was loaded by libharkio2.
1 heights, 72 directions, 1 ranges, 7 microphones, 512 points
Source 0 is created.
 (skipped)
Source 5 is removed.
UINodeRepository::Scan()
 (skipped)

14.3.1.2 Checking the results

After you ran the script, you will find two kinds of text files: result_*.txt and log_*.txt. If you cannot find them, you failed the execution. Check the following things:

1. Check if f101_000.wav, f101_090.wav, f101_180.wav and f101_270.wav are in the ../data directory. These files are simulation synthetic sounds of the 8ch sound recording audio waveforms to which impulse responses from the 0, 90, 180 and 270 degrees direction are superimposed. They are interpreted as sounds obtained in the case sounds are given to a virtual robot from each direction. Sound source localization results are not displayed without these files.

2. Check if the music.dat files are in the ../config directory. They are the impulse response files of a virtual robot. Sound source localization results are not displayed without these files.

result_*.txt contain sound source localization results generated by SaveSourceLocation in a text format. They contains the frame number and sound source direction for each frame. You can load them using LoadSourceLocation . See HARK document for the format of the file.

log_*.txt contains the MUSIC spectrum generated by LocalizeMUSIC whose DEBUG property is true. MUSIC spectrum means a kind of confidence of the sound existence, calculated for each time and direction. The sound exists if the MUSIC spectrum value is high. See LocalizeMUSIC in HARK document for details. We also provide the visualization program of MUSIC spectrum. Run the following commands:

> python plotMusicSpec.py log_microcone_90deg.txt 
> python plotMusicSpec.py log_microcone_0deg.txt 
> python plotMusicSpec.py log_microcone_moving.txt 

If you fail, your system may not have necessary software. Run the following command:

For Ubuntu sudo apt-get install python python-matplotlib

This result can be used for tuning the THRESH property of SourceTracker .

14.3.1.3 Sample network description

This sample has nine nodes. Three nodes are in MAIN (subnet) and six nodes are in MAIN_LOOP (iterator). MAIN (subnet) and MAIN_LOOP (iterator) are shown in Figures 14.7 and 14.8. AudioStreamFromWave loads the waveform, MultiFFT transforms it to spectrum, LocalizeMUSIC localizes the sound, SourceTracker tracks the localization result, then, DisplayLocalization shows them and SaveSourceLocation stores as a file.

Figure 14.7: MAIN (subnet)

Figure 14.8: MAIN_LOOP (iterator)

Table 14.10 summarizes the main parameters. The most important parameter is A_MATRIX, which specifies a file name of transfer function for localization. If you use a microphone array we support, you can download them from HARK web page. However, if you want use your own microphone array, you need to make it by harktool.