HARK version 1.1.0 Document : Application to telepresence robot

1.3.4 Application to telepresence robot

In March 2010, HARK and a system that visualizes sound environment was transplanted to the telepresence robot Texai made by Willow Garage, the US, and we realized a function that enables remote users to display sound source directions in pictures and listen to the sound from the sound source in a specific direction ¹. The design of acoustic information presentation in a telepresence robot is based on the past experience “Auditory awareness is the key technology” described above.

$\includegraphics[width=.6\linewidth ]{fig/Intro/Texai-2people-1texai.eps}$

Figure 1.10: A remote operator interacts with two speakers and one Texai through Texai (center). This demonstration was performed in California (CA), Texai on the left was operated from Indiana (IN) by remote control.

Details on transplant of HARK and development of an related modules of HARK for Texai are separated to the following two processes.

Installation of microphones in Texai, measurement of impulse response and installation of HARK,
Implementation of HARK interfaces and modules to ROS (Robot Operating System), which Texai control programs runs on.

$\includegraphics[width=\linewidth ]{fig/Intro/Texai-1st-head.eps}$

Figure 1.11: Closeup of the first head of Texai Eight MEMS microphones are embedded on a disk

$\includegraphics[width=.8\linewidth ]{fig/Intro/newTexai.eps}$

Figure 1.12: Closeup of the head of Texai Eight MEMS microphones are embedded in the shape of circle

Figure 1.11 shows the microphones firstly embedded. This robot was placed in the lecture room and dining hall where it was going to be used, and impulse responses were measured every five degrees and performance of source localization was estimated in each place. Next, in order to improve its visual impression and further to reduce the cross-talks between microphones, we discussed to put a head on Texai. Concretely, it was a bamboo-made salad bowl available in general shops. MEMS microphones were embedded on the points where the diameter became same as that of the head firstly set (FigurefignewTexai). Impulse responses were measured every five degrees and performance of source localization was estimated in the same way as above. The result revealed that there was not much difference in their performance.

$\includegraphics[width=.6\linewidth ]{fig/Intro/remoteDisplay.eps}$

Figure 1.13: Image seen by the remote operator through Texai

As for GUI, the overview and filter of Visual Information-seeking matra were implemented. It is shown in Figure 1.13 The arrows from the center in the all-round view, obliquely down Texai itself, are the sound source direction of the speaker. Length of the arrow indicates sound volume. The figure shows three speakers speaking. An image from another camera on Texai is shown in the lower-right corner and that from the remote operator is shown in the lower left corner. The circular arc in the figure indicates the range for filtering. The sound that received from directions within this arc is sent to the remote operator. As shown in Figure 1.14, data is transmitted through the Internet.

$\includegraphics[width=\linewidth ]{fig/Intro/Texai-dataflow.eps}$

Figure 1.14: Teleoperation of Texai

$\includegraphics[width=\linewidth ]{fig/Intro/Texai-modifiedDataflow.eps}$

Figure 1.15: Built-in of HARK in Texai

Since the control command groups for remote operators and GUI are all implemented as RS modules, HARK was transplanted in the way shown Figure 1.15. The brown part in the figure is the HARK system. The modules developed here are available at the website of ROS. These series of works including processing of the head, measurement of impulse response, pretests and design of GUI and control command groups were successfully completed in one week. We presume that the high modularity of HARK and ROS contributed to the improvement of productivity.

Footnotes

http://www.willowgarage.com/blog/2010/03/25/hark-texai