MVTEC HALCON
In order to realize a human-friendly interface, it is necessary that an intelligent room finds a person autonomously and understands his or her gesture. So as to make the procedures easier and more robust, we set a CCD camera at the ceiling
We focus on the gesture by a hand. First the system finds a person with the CCD camera, and then extracts the hand region using color information. Then, the finger regions are extracted from the hand region and the number of the fingers is recognized. Pointing can be defined that the number of extracted fingers is one and the hand is not moving.
We recognize a person by detecting his or her head. In order to reduce calculation cost, the search area of the hand is set up from the center of a head region. We convert an image to hsv space to decrease the influence of brightness, and extract a hand region in the 2-dimensional space of hue and saturation.
The number of fingers serves as the important element of the method of intention transfer. The procedure for recognizing the fingers is illustrated in Fig.2. The extracted hand region (1) is reduced and finger regions are eliminated (2). Then the region is expanded (3) to the original size. The difference of two regions (1)-(3) represents finger regions (4). It is considered that the number of finger regions is the number of fingers.
We suppose that an object is pointed when the number of fingers is one and the hand stops. The direction of the principal axis of the finger region is obtained as the pointing direction. Suppose that the positions of target objects in the room are given. Then the direction for each object from the hand is calculated. The angle between the direction of each object and obtained pointing direction is calculated. The object with smallest angle is recognized as the pointed object, if the angle is less than some threshold value. For example, object A is recognized as the pointed object in Fig.3.
Our experimental system consists of a PC (Pentium III 500MHz, Windows NT), image processing board PXC200, a color CCD camera attached at the ceiling, and the image processing software HALCON.
The experiment for extracting the hand region was performed as shown in Fig.4. Head region and hand region are extracted, and the number of fingers is recognized as one.
In our experiments with 200 samples for three subjects the recognition rates for the number of fingers (0-5) are as high as 96.0% on average (Fig.5).
The experiments to point to two objects were performed for 200 images as shown in Fig.6 and 7. The angle between the objects was set to 60°, 45°, 30° and 15°, and the average recognition rate was 87.1%. When the angle was larger than 30°, the recognition was almost correct.
So far, methods for recognizing the number of fingers and the pointing direction were developed and evaluated by experiments. The future works include the construction of a more practical gesture recognition system, and an intelligent room itself.
Author: Mr. Yuichi Nakanishi, Mr. Takashi Yamagishi and Prof. Kazunori Umeda of Chuo University, Japan
All product names, trademarks and images of the products/trademarks are copyright by their holders. All rights reserved.
Published on: November 13, 2020
