3.3.1 FlowDesigner for Linux

An overview of FlowDesigner is shown in Figure3.4 FlowDesigner is started using the following command.

> flowdesigner

3.3.1.1 Basic operation of FlowDesigner

A node is a processing unit that processes data. First, an example of a node is shown in Figure 3.5. The green part enclosed with a rectangle is the node, and the node’s name is displayed in the center. In this example, PostFilter is displayed. The name node_Postfilter_1 displayed below the node is an instance name of the node. The same node, when applied multiple times, is named after the order in which it is created. For example, node_Postfilter_2, node_Postfilter_3. It is useful to distinguish each node. Each node may have multiple terminals for inputs and outputs of data processed. The black points on the right and left of the node are the terminals. Those on the left are input terminals and on the right are output terminals. Terminals are named or unnamed depending on the cases. All terminals are named in this example. A node has at least one output terminal though there are no input terminals in some cases. For example, the constant node included in FlowDesigner by default does not have inputs but one output.

Data processing flows are defined by positioning multiple nodes and connecting the terminals of the nodes. A series of connected nodes is called a network, and creating a network itself is programming. Networks can be saved using the file save menu. Once a network is saved, it can be loaded by FlowDesigner anytime. When opening a saved file with a text editor, an XML-like description is seen. The user may edit such a file directly by a text editor once they get used to HARK.

Since some nodes have properties, the user can modify the processing of the nodes in detail by setting their property values. To set property values, move the mouse cursor over a node and left-double click on it. The setting DIALOG window opens and the user may input property values using the keyboard, or select values. Since there are data types for property values, the user is required to set data types and values as a pair. For details of the data types, see the chapter Data Type. Select data types from the pull-down menu. For values, the user may enter them with the keyboard or select values from the pull-down menu depending on the case.

3.3.1.2 Module operation / terminal connection / property setting

The following describes details of node operations, terminal connections and property setting.

• Creating a node
  Right-click on the network construction tab to display a list of node categories. Choose a category and left click on your desired node. To cancel the addition of the node, click the background part of the network construction tab (either of right or left).

• Moving a node
  To relocate a node to an easy-to-see position, click and drag the node with the left mouse button.

• Deleting a node
  Right-click on the node and select "delete".

• Copying a node
  Hold down the Shift key and left-click the node to copy it. Since attribute values are copied along with the node itself, this function is useful in constructing a network that executes the same processes in parallel.

• Connecting a node
  Modules are connected by connecting the terminals with arrows. Left-click on the source terminal, drag and release it on the connection point. The user can connect input to output terminals but cannot connect between input terminals and between output terminals. Multiple connections can be drawn from one output terminal to send the result of a node to multiple nodes. On the other hand, multiple connections cannot be established onto one input terminal.

  In some cases, input and output terminals cannot be established, for instance when the terminal cannot process a particular data type. Such a connection is indicated with a red arrow. In some cases, even though the arrow is black, an error may occur in a program execution when failing to judge the data type the terminal can process. Since the automatic data type judgment function is only a supplementary tool, it is recommended for the user to confirm appropriate data types in the node reference for accurate connections.

• Disconnecting a node
  Hold down Shift and right-click on the starting point or end point of the arrow of the connection that the user wish to disconnect.

• Bending a node connection line
  When moving a node, the node moves with the connection maintained. When positioning a large number of nodes, nodes and their connection lines overlap, which makes them difficult to see. Therefore, it is recommended to bend the lines so that network can be easily seen with fewer overlaps. Left-click the connection line and drag it to bend the line at that point.

• Setting a property value
  Right-click on a node and select “Properties" to display its settings dialog. Enter your desired values and data types, referring to the node reference, and click on “Apply". The settings are saved and the dialog is closed by clicking on “ OK".

Next, we describe how to create a network, using the example of a sound source localizing network.

3.3.1.3 Creating a network for beginners

This section shows an example of building a sound source localizing network for users who are creating such a network for the first time. As well as the main network tab, the subnetwork tab for Iteration is needed. First, we will build a sound source waveform reader in the main tab, then create a subnetwork and build localization and result display parts in the subnetwork.

We first describe the modular arrangement of the sound source waveform reading part, as well as its connections and property settings. Start FlowDesigner, and the MAIN (subnet) tab will appear on the right side of the window. Proceed to work in this tab first. Add the Constant node and the InputStream node as shown in Figure 3.7. Right-click on the grey background in the tab, and a pull-down menu will appear. Move the mouse cursor onto New Node, and the list of nodes registered in FlowDesigner will be displayed. Here, select the General category, and the list of nodes registered in General will be displayed. The Constant node is included in this list. From there, move the mouse cursor onto Constant to add a Constant node in the MAIN (subnet) tab. In the following descriptions, an operation to select this node out of a pull-down menu is indicated as “New Node $\rightarrow $ General $\rightarrow $ Constant". In the same way, add the InputStream node as New Node $\rightarrow $ IO $\rightarrow $ InputStream.

Next, connect the Constant node and an InputStream node as shown in Figure3.8.

Next, set properties of the node. Here, set only the Constant node. The InputStream node does not have settable property values. Right-click on the Constant node and select Properties to reveal the property setting dialog. The dialog contains Parameters, Comments and Inputs/Outputs tabs, with the Parameters tab open by default. The Comments tab and Inputs/Outputs tab are not used. The Parameters tab contains three items: NAME, Type and Value. NAME is for a property name, Type indicates a data type of values to be set, Value indicates the value to be designated for attributes. The Constant node has only one settable property, which has the attribute name VALUE. Since a file name is to be indicated in this node, give tutorial1.wav for the file name in the property value. The data type is string . Finally, save the settings by pressing “Apply" in the "Properties" window, followed by "Close". Alternatively, pressing “OK” to perform these two operations simultaneously. Since the user cannot operate other windows of FlowDesigner while the "Properties" window is displayed, it is necessary to finish the property setting before returning to such operations. Here, the property setting is complete, and a network to read from a sound source waveform has been created.

Next, we describe subnetwork creation. A subnetwork is created by selecting "Add Iterator" from the "Networks" menu in FlowDesigner. Select “Add Iterator" to enter a name for the Iterator tab. If using the default name "LOOP0", press "OK". Subnetwork creation is cancelled by pressing the cancel button. Clicking on "OK" for LOOP0, a tab named "LOOP0" appears next to the MAIN (subnet) tab and the LOOP0 tab becomes active. Build a localization part and result display part in this tab. This sound source localization subnetwork is built with nodes for file reading, FFT, sound source localization using the MUSIC method, sound source tracking and localization result display. Figure 3.9 shows nodes added using the following procedures.

New Node -> HARK -> AudioIO      -> AudioStreamFromWave
New Node -> HARK -> MISC         -> MultiFFT 
New Node -> HARK -> Localization -> LocalizeMUSIC 
New Node -> HARK -> Localization -> SourceTracker 
New Node -> HARK -> Localization -> DisplayLocalization 

Connect these nodes as shown in Figure 3.10.

After completing the connection, set node properties, assuming the audio file to be processed is 16,000 [Hz]. In the AudioStreamFromWave node, there are three property values: LENGTH, ADVANCE and USE_WAIT. LENGTH and ADVANCE indicate the analysis frame length and frame shift length of speech by unit samples, respectively. The data type is int . Default values 512 and 160 are set, which correspond to 32 msec and 10 msec, respectively. The user does not need to change these values. The default value falseis set in USE_WAIT of the data type bool . The user does not need to change these values.

In the MultiFFT node, there are three properties to set: LENGTH, WINDOW and WINDOW_LENGTH. The analysis frame length, window function type and window length of the sound are expressed in unit samples. The default values 512, CONJ and 512 are set. LENGTH and WINDOW_LENGTH correspond to 32 msec and its data type is int . CONJ designates the user to use the CONJ window as an analysis window. Its data type is string . The user does not need to change these values. LocalizeMUSIC nodes have nineteen properties to set: MUSIC_ALGORITHM, TF_CHANNEL_SELECTION, LENGTH, SAMPLING_RATE, A_MATRIX, ELEVATION, WINDOW, PERIOD, NUM_SOURCE, MIN_DEG, MAX_DEG, LOWER_BOUND_FREQUENCY, UPPER_BOUND_FREQUENCY, SPECTRUM_WEIGHT_TYPE, A_CHAR_SCALING, MANUAL_WEIGHT_SPLINE, MANUAL_WEIGHT_SQUARE, ENABLE_EIGENVALUE_WEIGHT, and DEBUG. The details of those parameters are given in the Parameters section in LocalizeMUSIC . The example of the parameter setting is shown below.

• MUSIC_ALGORITHM : SEVD

• TF_CHANNEL_SELECTION : <Vector<int> 0 1 2 3 4 5 6 7> 

• LENGTH : 512[samples]

• SAMPLING_RATE : 16000[Hz]

• A_MATRIX : Filename of your transfer function database

• ELEVATION : 16.7[deg]

• WINDOW : 50[frames]

• PERIOD : 50[frames]

• NUM_SOURCE : 2[sources]

• MIN_DEG : -180[deg]

• MAX_DEG : 180[deg]

• LOWER_BOUND_FREQUENCY : 500[Hz]

• UPPER_BOUND_FREQUENCY : 2800[Hz]

• SPECTRUM_WEIGHT_TYPE : Uniform

• A_CHAR_SCALING : Not displayed

• MANUAL_WEIGHT_SPLINE : Not displayed

• MANUAL_WEIGHT_SQUARE : Not displayed

• ENABLE_EIGENVALUE_WEIGHT : true

• DEBUG : false

For the DisplayLocalization node, there is one property for the value of LOG_IS_PROVIDED. To perform logging of the source locations, set true. Its data type is bool , and the default value is false. The user does not need to change these values.

Next, set Iterator. Iterator can be considered as a subroutine of MAIN (subnet) and requires INPUT, OUTPUT and CONDITION. INPUT and OUTPUT can be likened to the input and output of the subroutine. Iterator repeatedly performs processing while it is the subroutine; therefore, the user needs to describe CONDITION, which is the loop’s halt condition. Set it as shown in Figure 3.11. To set INPUT, left-click on the input terminal while holding down Shift. Press “OK" and “INPUT" will be displayed on the input terminal in red letters. To set OUTPUT, left-click on the input terminal while holding down Shift. Press “OK," and “OUTPUT" will be displayed on the terminal in blue letters. CONDITION is set by left-clicking on the output terminal while holding down Control. “CONDITION" is displayed in purple. When setting INPUT, OUTPUT or CONDITION in an unintended location, the user may cancel the setting by left-clicking on those letters while pressing Shift.

Finally, integrate Iterator and Subnet. Return to the network display of MAIN (subnet) by pressing the MAIN (subnet) tab. Left-click on the grey background and move the mouse cursor onto New Node; a new item named subnet should have appeared on the node category name, with an item named LOOP0. Left-click on LOOP0, similar to when creating a node, and a node named LOOP0 with one input and one output will be added. In other words, the subnetwork described in LOOP0 (Iterator) is made with this virtual node. To complete the source localization network, connect InputStream to LOOP0 and set OUTPUT on LOOP0 as shown in Figure 3.12.