HARK version 1.1.0 Document : LocalizeMUSIC

6.2.13 LocalizeMUSIC

Outline of the node

From multichannel speech waveform data, direction-of-arrival (DOA) in the horizontal plane is estimated using the MUltiple SIgnal Classification (MUSIC) method. It is the main node for Sound Source Localization in HARK .

Necessary file

The transfer function file which consists of a steering vector is required. It is generated based on the positional relationship between the microphone and sound, or the transfer function for which measurement was performed.

Usage

This node estimates a sound’s direction and amount of power using the MUSIC method. Detection of a direction with high power in each frame allows the system to know the direction of sound, the number of sound sources, the speech periods, etc. to some extent. The orientation result outputted from this node is used for post-processing such as tracking and source separation.

Typical connection

Figure 6.22 shows a typical connection example.

$\includegraphics[width=0.85\linewidth ]{fig/modules/LocalizeMUSIC.eps}$

Figure 6.22: Connection example of LocalizeMUSIC

Input-output and property of the node

Input

INPUT: Matrix<complex<float> > , Complex frequency representation of input signals with size $M \times (NFFT/2+1)$ .
NOISECM: Matrix<complex<float> > type. The correlation matrix for each frequency bin. The $NFFT/2 + 1$ correlation matrices are inputted, corresponding to the $M$ -th complex square matrix. The rows of Matrix<complex<float> > express frequency ( $NFFT / 2+1$ rows) and the columns express the complex correlation matrix ( $M * M$ columns). This input terminal can also be left disconnected; then an identity matrix is used for the correlation matrix instead.

Output

OUTPUT: Source position (direction) is expressed as Vector<ObjectRef> type. ObjectRef is a Source and is a structure which consists of the power of the MUSIC spectrum of the source and its direction. The element number of Vector is a sound number ( $N$ ). Please refer to node details for the details of the MUSIC spectrum.
SPECTRUM: Vector<float> type. Power of the MUSIC spectrum for every direction. This output terminal is not displayed by default.
SPECTRUM_BIN: Matrix<float> type. The Power of the MUSIC spectrum for every frequency bin and for every direction is outputted as a 2 dimensional array. The SPECTRUM output terminal outputs those which are added to the frequency axis. This output terminal is not displayed by default.
EIGVAL_BIN: Matrix<float> type. The eigenvalue (or singular value) computed from the eigenvalue decomposition (or singular value decomposition) of the correlation matrix for every frequency bins is outputted. The row expresses the frequency bin ( $NFFT/2+1$ ), and the column expresses the degree ( $M$ ) of the correlation matrix. This output terminal is not displayed by default.

Refer to Figure 6.23 for the addition method of hidden output.

$\includegraphics[width=\linewidth ]{fig/modules/LocalizeMUSIC_output1.eps}$
Step 1 Right-click LocalizeMUSIC and click Properties.

$\includegraphics[width=\linewidth ]{fig/modules/LocalizeMUSIC_output2.eps}$
Step 2 Click Inputs/Outputs.

$\includegraphics[width=\linewidth ]{fig/modules/LocalizeMUSIC_output3.eps}$
Step 3 Enter SPECTRUM in the input form of Outputs, click Add.

$\includegraphics[width=\linewidth ]{fig/modules/LocalizeMUSIC_output4.eps}$
Step 4 Check that SPECTRUM has been added to the output list of Outputs, click O.K.

$\includegraphics[width=\linewidth ]{fig/modules/LocalizeMUSIC_output5.eps}$
Step 5 The SPECTRUM output terminal is added to the node.

Figure 6.23: Usage example of hidden outputs Display of SPECTRUM terminal

Table 6.26: Parameter list of LocalizeMUSIC

Parameter name	Type	Default value	Unit	description
MUSIC_ALGORITHM	`string`	SEVD		Algorithm of MUSIC
TF_CHANNEL_SELECTION	`Vector<int>`	See below.		Channel number used
LENGTH	`int`	512	[pt]	FFT points ( $NFFT$ )
SAMPLING_RATE	`int`	16000	[Hz]	Sampling rate
A_MATRIX	`string`			Transfer function file name
ELEVATION	`float`	16.7	[deg]	Elevation angle of sound
WINDOW	`int`	50	[frame]	Frames to normalize CM
PERIOD	`int`	50	[frame]	The cycle to compute SSL
NUM_SOURCE	`int`	2		Number of sounds
MIN_DEG	`int`	-180	[deg]	Minimum azimuth
MAX_DEG	`int`	180	[deg]	Maximum azimuth
LOWER_BOUND_FREQUENCY	`int`	500	[Hz]	Lower bound frequency
UPPER_BOUND_FREQUENCY	`int`	2800	[Hz]	Upper bound frequency
SPECTRUM_WEIGHT_TYPE	`string`	Uniform		Type of frequency weight
A_CHAR_SCALING	`float`	1.0		Coefficient of weight
MANUAL_WEIGHT_SPLINE	`Matrix<float>`	See below.		Coefficient of spline weight
MANUAL_WEIGHT_SQUARE	`Matrix<float>`	See below.		Key point of rectangular weight
ENABLE_EIGENVALUE_WEIGHT	`bool`	`true`		Enable eigenvalue weight
DEBUG	`bool`	`false`		ON/OFF of debug output

Parameter

MUSIC_ALGORITHM: string type. Selection of algorithm used in order to calculate the signal subspace in the MUSIC method. SEVD represents standard eigenvalue decomposition, GEVD represents generalized eigenvalue decomposition, and GSVD represents generalized singular value decomposition. LocalizeMUSIC enters a correlation matrix with sound information from the NOISECM terminal, and possesses a function which can do SSL whitening of the noise (suppression). SEVD realizes SSL without the function. When SEVD is chosen, the input from NOISECM terminal is disregarded. Although both GEVD and GSVD have a function to whiten the noise inputted from the NOISECM terminal, GEVD has better noise suppression performance compared with GSVD. It has the problem that the calculation time takes approximately 4 times longer. Depending on the scene and computing environment, you can select one of the three algorithms. Please refer to node details for the details of algorithm.
TF_CHANNEL_SELECTION: Vector<int> type. Of steering vectors of multichannel stored in the transfer function file, it is parameters which chooses the steering vector of specified channel to use. The channel number begins from 0 like ChannelSelector . Signal processing of 8 channel is assumed by default and it is set as <Vector<int> 0 1 2 3 4 5 6 7> . It is necessary to align the number ( $M$ ) of elements of the parameters with the channel number of incoming signals. Moreover, it is necessary to align the order of channel and the channel order of TF_CHANNEL_SELECTION to be inputted into INPUT terminal.
LENGTH: int type. 512 is the default value. FFT point in the case of fourier transform. It is necessary to align it with the FFT points to the preceding paragraph.
SAMPLING_RATE: int type. 16000 is the default value. Sampling frequency of input acoustic signal. It is necessary to align with other nodes like LENGTH.
A_MATRIX: string type. There is no default value. The file name of the transfer function file is designated. Both absolute path and relative path are supported. Refer to the harktool3 for the creation method of the transfer function file.
ELEVATION: float type. Elevation angle of sound is specified. At the present, HARK does not support estimation of elevation angle of sound, thus, it is taken as the fixed value. For the default value, supposing the position relationships of the robot with human, as one example, when supposing that the height of microphone array of the robot is 1.2 [m] and the position of a person is 1.5 [m], and the distance between a robot and person is about 1.0 [m], it was set up as $\arctan \frac{1.5-1.2}{1.0}$ [rad] $= 16.7$ [deg].
WINDOW: int type. 50 is the default value. The number of smoothing frames for correlation matrix calculation is designated. Within the node, the correlation matrix is generated for every frame from the complex spectrum of the input signal, and the addition mean is taken by the number of frames specified in WINDOW. Although the correlation matrix will be stabilized if this value is enlarged, time delays become long due to the long interval.
PERIOD: int type. 50 is the default value. The cycle of SSL calculation is specified in frames number. If this value is large, the time interval for obtaining the orientation result becomes large, which will result in improper acquisition of the speech interval or bad tracking of the mobile sound. However, since the computational cost increases if it is small, tuning according to the computing environment is needed.
NUM_SOURCE: int type. 2 is the default value. It is the number of dimensions of the signal subspace in the MUSIC method, and can be practically interpreted as the number of desired sound sources to be emphasized in the peak detection. It is expressed as $N_ s$ in the following nodes details. It should be $1 \leq N_ s \leq M - 1$ . It is desirable to match the sound number of the desired sound, but, for example, in the case of the number of desired sound sources being 3, the interval that each sound pronounces is different, thus, it is sufficient to select a smaller value than it is practically.
MIN_DEG: int type. -180 is the default value. It is the minimum angle for peak search, and is expressed as $\theta _{min}$ in the node details. 0 degree is the robot front direction, negative values are the robot right hand direction, and positive values are the robot left-hand direction. Although the specified range is considered as $\pm 180$ degrees for convenience, since the surrounding range of 360 degrees or more is also supported, there is no particular limitation.
MAX_DEG: int type. 180 is the default value. It is the maximum angle for peak search, and is expressed as $\theta _{max}$ in the node details. Others are the same as that of MIN_DEG.
LOWER_BOUND_FREQUENCY: int type. 500 is the default value. It is the minimum of frequency bands which is taken into consideration for peak detection, and is expressed as $\omega _{min}$ in the node details. It should be $0 \leq \omega _{min} \leq {\rm SAMPLING\_ RATE} / 2$ .
UPPER_BOUND_FREQUENCY: int type. 2800 is the default value. It is the maximum of frequency bands Which is taken into consideration for peak detections, and, is expressed as $\omega _{max}$ below. It should be $\omega _{min} < \omega _{max} \leq {\rm SAMPLING\_ RATE} / 2$ .
SPECTRUM_WEIGHT_TYPE: string type. ‘Uniform’ is the default value. The distribution of weights against the frequency axial direction of the MUSIC spectrum used for peak detections is designated. ‘Uniform’ sets weights to OFF. ‘A_Characteristic’ gives the MUSIC spectrum weights imitating the sound pressure sensitivity of human hearing. ‘Manual_Spline’ gives the MUSIC spectrum weights suited to the Cubic spline curve for which the point specified in MANUAL_WEIGHT_SPLINE is considered as the interpolating point. ‘Manual_Square’ generates the rectangular weights suited to the frequency specified in MANUAL_WEIGHT_SQUARE, and gives it to MUSIC spectrum.
A_CHAR_SCALING: float type. 1.0 is the default value. This is scaling term which modifies the A characteristic weight on the frequency axis. Since the A characteristic weight imitates the sound pressure sensitivity of human’s hearing, filtering to suppress sound outside of the speech frequency range is possible. Although the A characteristic weight has a standard, depending on the general sound environment, noise may enter the speech frequency range, and it may be unable to orientate well. Then, the A characteristic weight should be increased, causing more suppression, especially in the lower frequencies.
MANUAL_WEIGHT_SPLINE: Matrix<float> type.
<Matrix<float> <rows 2> <cols 5> <data 0.0 2000.0 4000.0 6000.0 8000.0 1.0 1.0 1.0 1.0 1.0> > is the default value. It is designated with the float value 2-by- $K$ matrix. $K$ is equivalent to the number of interpolation points for spline interpolations. The first row specifies the frequency and the second row specifies the weight corresponding to it. Weighting is performed according to the spline curve which passes along the interpolated point. By default, the weights are all set to 1 for the frequency bands from 0 [Hz] to 8000 [Hz] .
MANUAL_WEIGHT_SQUARE: Vector<float> type. <Vector<float> 0.0 2000.0 4000.0 6000.0 8000.0> is the default value. By the frequency specified in MANUAL_WEIGHT_SQUARE, the rectangular weight is generated and is given to MUSIC spectrum. For the frequency bands from the odd components of MANUAL_WEIGHT_SQUARE to the even components, the weight of 1 is given, and for the frequency bands from the even components to the odd components, the weight of 0 is given. By default, the MUSIC spectrum from 2000 [Hz] to 4000 [Hz] and 6000 [Hz] to 8000 [Hz] can be suppressed.
ENABLE_EIGENVALUE_WEIGHT: bool type. True is the default value. For true, in the case of calculation of the MUSIC spectrum, the weight is given as the square root of the maximum eigenvalue (or the maximum singular value) acquired from eigenvalue decomposition (or singular value decompositions) of the correlation matrix. Since this weight greatly changes depending on the eigenvalue of the correlation matrix inputted from NOISECM terminal when choosing GEVD and GSVD for MUSIC_ALGORITHM, it is good to choose false.
DEBUG: bool type. ON/OFF of the debug output and the format of the debug output are as follows. First, the set of index of sound, direction, and power is outputted in tab delimited for only several number of sound detected in frames. ID is the number given for convenience in order from 0 for every frame, though the number itself is meaningless. For direction [deg], an integer with rounded decimal is displayed. As for power, the power value of MUSIC spectrum $\bar{P}(\theta )$ of Eq. () is outputted as is. Next, “MUSIC spectrum” is outputted after a line feed, and the value of $\bar{P}(\theta )$ of Eq. () is displayed for all $\theta$ .

Details of the node

The MUSIC method is the method of estimating the direction-of-arrival (DOA) utilizing the eigenvalue decomposition of the correlation matrix among input signal channels. The algorithm is summarized below.

Generation of transfer function

In the MUSIC method, the transfer function from sound to each microphone is measured or calculated numerically and it is used as a priori information. If the transfer function in the frequency domain from sound $S(\theta )$ in direction $\theta$ in view of microphone array to the $i$ -th microphone $M_ i$ is set to $h_ i(\theta ,\omega )$ , the multichannel transfer function multichannel can be expressed as follows.

$\begin{equation} \label{eqtf} {\bm@general \boldmath \m@ne \mv@bold \bm@command H}(\theta ,\omega ) = [h_1(\theta ,\omega ),\cdots ,h_ M(\theta ,\omega )] \end{equation}$

(8)

This transfer function vector is prepared for every suitable interval delta, $\Delta \theta$ (non-regular intervals are also possible) by calculation or measurement in advance. In HARK , harktool3 is offered as a tool which can generate the transfer function file also by numerical calculation and also by measurement. Please refer to the paragraph of harktool3 for the prepare a specific transfer function file. In the LocalizeMUSIC node, this a priori information file (transfer function file) is imported and used with the file name specified in A_MATRIX. Thus, since the transfer function is prepared for every direction of sound and is scanned to the direction using the direction vector (or transfer function, in the case of orientation), it is sometimes called ‘steering vector’.

Calculation of correlation matrix between the inputs signal channels

The operation by HARK begins from here. First, the signal vector in the frequency domain obtained by short-time fourier transform of the input acoustic signal in $M$ channel is found as follows.

$\begin{equation} {\bm@general \boldmath \m@ne \mv@bold \bm@command X}(\omega ,f) = [X_1(\omega ,f), X_2(\omega ,f), X_3(\omega ,f), \cdots , X_ M(\omega ,f)]^ T, \label{eqLocalizeMUSIC_ cor} \end{equation}$

(9)

where $\omega$ expresses frequency and $f$ expresses frame index. In HARK , the process so far is performed by the MultiFFT node in the preceding paragraph.

The correlation matrix between channels of the incoming signal ${\bm@general \boldmath \m@ne \mv@bold \bm@command X}(\omega ,f)$ can be defined as follows for every frame and for every frequency .

$\begin{equation} {\bm@general \boldmath \m@ne \mv@bold \bm@command R}(\omega ,f) = {\bm@general \boldmath \m@ne \mv@bold \bm@command X}(\omega ,f){\bm@general \boldmath \m@ne \mv@bold \bm@command X}^*(\omega ,f) \label{eqLocalizeMUSIC_1} \end{equation}$

(10)

where $()^*$ represents the conjugate transpose operator. If this ${\bm@general \boldmath \m@ne \mv@bold \bm@command R}(\omega ,f)$ is utilized in following processing as is, theoretically, it will be satisfactory, but practically, in order to obtain the stable correlation matrix, those time averaging is used in HARK .

$\begin{equation} {\bm@general \boldmath \m@ne \mv@bold \bm@command R}’(\omega ,f) = \frac{1}{{\rm WINDOW}}\sum _{i=0}^{{\rm WINDOW}-1}{\bm@general \boldmath \m@ne \mv@bold \bm@command R}(\omega ,f+i) \label{eqLocalizeMUSIC_ period} \end{equation}$

(11)

Decomposition to the signal and noise subspace

In the MUSIC method, an eigenvalue decomposition or singular value decomposition of the correlation matrix ${\bm@general \boldmath \m@ne \mv@bold \bm@command R}’(\omega ,f)$ found in the Eq. () is performed and the $M$ -th space is decomposed into the signal subspace and the other subspace.

Since the processing has high computational cost, it is designed to be calculated only once in several frames. In LocalizeMUSIC , this operation period can be specified in PERIOD.

In LocalizeMUSIC , the method for decomposing into subspace can be specified by MUSIC_ALGORITHM.

When MUSIC_ALGORITHM is specified for SEVD, the following standard eigenvalue decomposition is performed.

$\begin{equation} {\bm@general \boldmath \m@ne \mv@bold \bm@command R}’(\omega ,f) = {\bm@general \boldmath \m@ne \mv@bold \bm@command E}(\omega ,f) {\bm@general \boldmath \m@ne \mv@bold \bm@command \Lambda }(\omega ,f) {\bm@general \boldmath \m@ne \mv@bold \bm@command E}^{-1}(\omega ,f)~ , \label{eqLocalizeMUSIC_ SEVD} \end{equation}$

(12)

where ${\bm@general \boldmath \m@ne \mv@bold \bm@command E}(\omega ,f)$ represents the matrix ${\bm@general \boldmath \m@ne \mv@bold \bm@command E}(\omega ,f) = [{\bm@general \boldmath \m@ne \mv@bold \bm@command e}_1(\omega ,f), {\bm@general \boldmath \m@ne \mv@bold \bm@command e}_2(\omega ,f), \cdots , {\bm@general \boldmath \m@ne \mv@bold \bm@command e}_ M(\omega ,f)]$ which consists of singular vectors which perpendicularly intersect each other, and ${\bm@general \boldmath \m@ne \mv@bold \bm@command \Lambda }(\omega )$ represents the diagonals matrix using the eigenvalue corresponding to individual eigenvectors as the diagonal component. In addition, the diagonal component of ${\bm@general \boldmath \m@ne \mv@bold \bm@command \Lambda }(\omega )$ , $[\lambda _1(\omega ), \lambda _2(\omega ),\dots ,\lambda _ M(\omega )]$ is considered to have been sorted in descending order.

When MUSIC_ALGORITHM is specified for GEVD, the following generalized eigenvalue decomposition is performed.

$\begin{equation} {\bm@general \boldmath \m@ne \mv@bold \bm@command K}^{-\frac{1}{2}}(\omega ,f){\bm@general \boldmath \m@ne \mv@bold \bm@command R}’(\omega ,f){\bm@general \boldmath \m@ne \mv@bold \bm@command K}^{-\frac{1}{2}}(\omega ,f) = {\bm@general \boldmath \m@ne \mv@bold \bm@command E}(\omega ,f) {\bm@general \boldmath \m@ne \mv@bold \bm@command \Lambda }(\omega ,f) {\bm@general \boldmath \m@ne \mv@bold \bm@command E}^{-1}(\omega ,f)~ , \label{eqLocalizeMUSIC_ GEVD} \end{equation}$

(13)

where ${\bm@general \boldmath \m@ne \mv@bold \bm@command K}(\omega ,f)$ represents the correlation matrix inputted from NOISECM terminal at the $f$ -th frame. Since large eigenvalues from the noise sources included in ${\bm@general \boldmath \m@ne \mv@bold \bm@command K}(\omega ,f)$ can be whitened (surpressed) using generalized eigenvalue decomposition with ${\bm@general \boldmath \m@ne \mv@bold \bm@command K}(\omega ,f)$ , SSL with suppressed noise is realizable.

When MUSIC_ALGORITHM is specified for GSVD, the following generalized singular value decomposition is performed.

$\begin{equation} {\bm@general \boldmath \m@ne \mv@bold \bm@command K}^{-1}(\omega ,f){\bm@general \boldmath \m@ne \mv@bold \bm@command R}’(\omega ,f) = {\bm@general \boldmath \m@ne \mv@bold \bm@command E}(\omega ,f) {\bm@general \boldmath \m@ne \mv@bold \bm@command \Lambda }(\omega ,f) {\bm@general \boldmath \m@ne \mv@bold \bm@command E}_ r^{-1}(\omega ,f)~ , \label{eqLocalizeMUSIC_ GSVD} \end{equation}$

(14)

where ${\bm@general \boldmath \m@ne \mv@bold \bm@command E}(\omega ,f), {\bm@general \boldmath \m@ne \mv@bold \bm@command E}_ r(\omega ,f)$ represents the matrix which consists of left singular vector and right singular vector, respectively, and ${\bm@general \boldmath \m@ne \mv@bold \bm@command \Lambda }(\omega )$ represents the diagonal matrix using each singular-value as the diagonal components.

Since the eigenvalue (or singular-value) corresponding to eigen vector space ${\bm@general \boldmath \m@ne \mv@bold \bm@command E}(\omega ,f)$ obtained by degradation has correlation with the power of sound, by taking eigen vector corresponding to the eigenvalue with the large value, only the subspace of loud desired sound with large power can be chosen. If the number of sounds to be considered is set to $N_ s$ , then eigen vector $[e_1(\omega ), \cdots , e_{N_ s}(\omega )]$ corresponds to the sound, are eigen vector $[e_{N_ s+1}(\omega ), \cdots , e_ M(\omega )]$ corresponds to noise. In LocalizeMUSIC , $N_ s$ can be specified as NUM_SOURCE.

Calculation of MUSIC spectrum

The MUSIC spectrum for SSL is calculated as follows using only noise-related eigen vectors.

$\begin{equation} P(\theta ,\omega ,f) = \frac{\left| {\bm@general \boldmath \m@ne \mv@bold \bm@command H}^*(\theta ,\omega ) {\bm@general \boldmath \m@ne \mv@bold \bm@command H}(\theta ,\omega ) \right|}{\sum _{i=N_ s+1}^ M \left| {\bm@general \boldmath \m@ne \mv@bold \bm@command H}^*(\theta ,\omega ) e_ i(\omega ,f) \right|} \label{eqLocalizeMUSIC_ music_ spectrum_ bin} \end{equation}$

(15)

In the denominator in the right-hand side, the inner product of the noise-related eigen vector and steering vector is calculated. On the space spanned by the eigen vector, since the noise subspace corresponding to small eigenvalue and the signal subspace corresponding to a large eigenvalue intersect perpendicularly each other, if the transfer function is a vector corresponding to the desired sound, this inner product will be 0 theoretically. Therefore, $P(\theta ,\omega ,f)$ diverges infinitely. In fact, although it does not diverge infinitely under the effect of noise etc., a sharp peak is observed compared to beam forming. The right-hand side of the numerator is an normalization term.

Since $P(\theta ,\omega ,f)$ is MUSIC spectrum obtained for every frequency, broadband SSL is performed as follows.

$\begin{equation} \bar{P}(\theta ,f) = \sum _{\omega =\omega _{min}}^{\omega _{max}} W_{\Lambda }(\omega ,f) W_{\omega }(\omega ,f) P(\theta ,\omega ,f)~ , \label{eqLocalizeMUSIC_ music_ spectrum} \end{equation}$

(16)

where $\omega _{min}$ and $\omega _{max}$ show the minimum and maximum of the frequency bands which are handled in the broadband integration of MUSIC spectrum, respectively, and they can be specified as LOWER_BOUND_FREQUENCY and UPPER_BOUND_FREQUENCY in LocalizeMUSIC , respectively.

Moreover, $W_{\Lambda }(\omega ,f)$ is the eigen-value weight in the case of broadband integration and is square root of the maximum eigenvalue (or maximum singular-value).

In LocalizeMUSIC , the presence or absence of eigenvalue weight can be chosen by ENABLE_EIGENVALUE_WEIGHT, and in case of false, it is $W_{\Lambda }(\omega ,f) = 1$ and in case of true, it is $W_{\Lambda }(\omega ,f) = \undefined {\lambda _1(\omega ,f)}$ . Moreover, $W_{\omega }(\omega ,f)$ is the frequency weight in the case of broadband integration, and the type can be specified as follows by SPECTRUM_WEIGHT_TYPE in LocalizeMUSIC .

In the case that SPECTRUM_WEIGHT_TYPE is Uniform
weights become uniform and $W_{\omega }(\omega ,f) = 1$ all frequency bins.
In the case that SPECTRUM_WEIGHT_TYPE is A_Characteristic
it will be A characteristic weight $\mathcal{W}(\omega )$ which the International Electrotechnical Commission standardizes. The frequency characteristics of A characteristic weight is shown in Figure 6.24. The horizontal axis is $\omega$ and the vertical axis is $\mathcal{W}(\omega )$ . In LocalizeMUSIC , the scaling term A_CHAR_SCALING of frequency direction is introduced to the frequency characteristic. If A_CHAR_SCALING is set as $\alpha$ , then the frequency weight actually used can be expressed as $\mathcal{W}(\alpha \omega )$ . In Figure 6.24, the case of $\alpha = 1$ and the case of $\alpha = 4$ are plotted as an example. The weight finally applied to the MUSIC spectrum is $W_{\omega }(\omega ,f) = 10^{\frac{\mathcal{W}(\alpha \omega )}{20}}$ . As an example, $W_{\omega }(\omega ,f)$ when A_CHAR_SCALING=1 is shown in Figure 6.25.
When SPECTRUM_WEIGHT_TYPE is Manual_Spline
it is the frequency weight in line with the curve in which the spline interpolation was carried out for the interpolating point specified in MANUAL_WEIGHT_SPLINE. MANUAL_WEIGHT_SPLINE is specified with the Matrix<float> type of 2-by- $k$ matrix. The first row represents the frequency and the second row represents the weight for the frequency. The interpolation mark $k$ may be any point. As an example, in the case that MANUAL_WEIGHT_SPLINE is
<Matrix<float> <rows 2> <cols 3> <data 0.0 4000.0 8000.0 1.0 0.5 1.0> >
the number of interpolation points is 3, and the spline curve to which the weight of 1, 0.5, and 1 is applied in three frequencies, 0, 4000, and 8000[Hz], on the frequency axis, respectively can be created. $W_{\omega }(\omega ,f)$ at that time is shown in Figure 6.26.
When SPECTRUM_WEIGHT_TYPE is Manual_Square
it is the frequency weight in line with the rectangular weight from which the rectangle changes at the frequency specified in MANUAL_WEIGHT_SQUARE. MANUAL_WEIGHT_SQUARE is specified in the $k$ -th Vector<float> type, and expresses the frequency to switch the rectangle. The number $k$ of the switching point is arbitrary. As an example, the rectangle weight $W_{\omega }(\omega ,f)$ in the case that MANUAL_WEIGHT_SQUARE is considered as
<Vector<float> 0.0 2000.0 4000.0 6000.0 8000.0>
is shown in Figure 6.27. By using this weight, two or more frequency domains which cannot be specified with only UPPER_BOUND_FREQUENCY and LOWER_BOUND_FREQUENCY can be chosen.

$\includegraphics[width=.5\linewidth ]{fig/modules/LocalizeMUSIC_AFilter_dB.eps}$

Figure 6.24: Frequency characteristic of characteristic weight when considering as SPECTRUM_WEIGHT_TYPE=A_Characteristic

$\includegraphics[width=.5\linewidth ]{fig/modules/LocalizeMUSIC_AFilter.eps}$

Figure 6.25: $W_{\omega }(\omega ,f)$ in case of SPECTRUM_WEIGHT_TYPE=A_Charasteristic and A_CHAR_SCALING=1

$\includegraphics[width=.5\linewidth ]{fig/modules/LocalizeMUSIC_Spline.eps}$

Figure 6.26: $W_{\omega }(\omega ,f)$ in case of SPECTRUM_WEIGHT_TYPE=Manual Spline

$\includegraphics[width=.5\linewidth ]{fig/modules/LocalizeMUSIC_Square.eps}$

Figure 6.27: $W_{\omega }(\omega ,f)$ in case of SPECTRUM_WEIGHT_TYPE=Manual_Square

Search of sound

Next, the local peak is detected from the range in $\theta _{min}$ to $\theta _{max}$ for $\bar{P}(\theta )$ of Eq. (), and the power of the MUSIC spectrum corresponding to DoA for the top $N_ s$ are outputted in descending order of the value. Moreover, the number of output sound sources may become below when the number of peaks does not reach to $N_ s$ . In LocalizeMUSIC , $\theta _{min}$ and $\theta _{max}$ can be specified in MIN_DEG and MAX_DEG, respectively.

Discussion

Finally, we describe the effect that whitening (noise suppression0 has on MUSIC spectrum in Eq. () when choosing GEVD and GSVD for MUSIC_ALGORITHM.

Here, as an example, consider the situation of four speakers (Directions = 75[deg], 25[deg], -25[deg], and -75[deg]) speaking simultaneously.

Figure 6.28(a) shows the result of choosing SEVD for MUSIC_ALGORITHM and not having whitened the noise. The horizontal axis is the azimuth, the vertical axis is frequency, and the value is $P(\theta ,\omega ,f)$ of the Eq. (). As shown in the figure, there is diffusion noise in the low frequency domain and -150 degree direction, which reveals that the peak is not correctly detectable to only the direction of the 4 speakers.

Figure 6.28(b) shows the MUSIC spectrum in the interval in which SEVD is chosen for MUSIC_ALGORITHM and 4 speakers do not perform speech. The diffusion noise and the direction noise observed can be seen in Figure 6.28(a).

Figure 6.28(c) is the MUSIC spectrum when generating ${\bm@general \boldmath \m@ne \mv@bold \bm@command K}(\omega ,f)$ from the information on Figure 6.28(b), choosing GSVD for MUSIC_ALGORITHM as general sound information, and whitening the noise. As shown in the figure, it can be seen that the diffusion noise and the direction noise contained in ${\bm@general \boldmath \m@ne \mv@bold \bm@command K}(\omega ,f)$ are suppressed correctly and the strong peaks are only in the direction of the 4 speakers.

Thus, it is useful to use GEVD and GSVD for known noise.

$\includegraphics[width=\linewidth ]{fig/modules/LocalizeMUSIC_Spectrum_SEVD.eps}$
(a) MUSIC spectrum when MUSIC_ALGORITHM=SEVD (four speakers)

$\includegraphics[width=\linewidth ]{fig/modules/LocalizeMUSIC_Spectrum_NOISE.eps}$
(b) MUSIC spectrum of the noise by generating ${\bm@general \boldmath \m@ne \mv@bold \bm@command K}(\omega ,f)$ (zero speaker)

$\includegraphics[width=\linewidth ]{fig/modules/LocalizeMUSIC_Spectrum_GSVD.eps}$
(c) MUSIC spectrum when MUSIC_ALGORITHM=GSVD (four speakers)

Figure 6.28: Comparison of MUSIC spectrum

References

F. Asano et. al, “Real-Time Sound Source Localization and Separation System and Its Application to Automatic Speech Recognition” Proc. of International Conference on Speech Processing (Eurospeech 2001), pp.1013–1016, 2001.
Toshiro Oga, Yutaka Kaneda, Yoshio Yamazaki, "Acoustic system and digital processing" The Institute of Electronics, Information and Communication Engineers.
K. Nakamura, K. Nakadai, F. Asano, Y. Hasegawa, and H. Tsujino, “Intelligent Sound Source Localization for Dynamic Environments”, in Proc. of IEEE/RSJ Int’l Conf. on Intelligent Robots and Systems (IROS 2009), pp. 664–669, 2009.