I. Introduction

The number of road signs in Korea is approximately 160,000, which are managed by up to 300 road management agencies. In 2001, a road sign management system was put in practice to manage and improve road signs in an efficient manner. However, it is difficult to build a database based on the instantaneous renewal of road sign data, because road signs are handled by various road management agencies and they change constantly. Constructing a road sign database involves two processes: general field surveys and the input/analysis of sign data, which cost 1.6 billion won and 5.2 billion won annually, respectively. Despite the difficulties of procuring suitable field survey equipment and manpower availability, efforts have been made to facilitate continuous management and maintenance. However, the data acquisition reliability has yet to reach 100% and it cannot be improved much more. Therefore, field survey techniques that utilize mobile mapping systems (MMSs) are required for the efficient construction of a road sign database using minimum human resources. Field surveys of road signs requires the acquisition of data from automatic images captured via a moving vehicle and their locations, which can be achieved using MMS-based applications.

The present study proposes a gradation image processing method for the development of a Road Sign Recognition Platform (RReP).

II. Review of Technology Trends

1. Studies of Other Sign Recognition

Studies of sign-related recognition have focused on road sign recognition and extraction techniques for traffic sign colors and shapes. Most color-based traffic sign extraction methods detect areas similar to a traffic sign using an image segmentation method based on the RGB, HIS and HSV color model [2][3][4]. However, the problem with these methods is that they fail to respond adaptively to environmental variation, such as illumination changes, because they assume that the color changes in images will not be significant. One solution to this problem is a sign detection method that employs a neural network for detection, where a self-organizing map filters candidate sign pixels from the color data after processing by color normalization. In other traffic sign detection methods, support vector machines are used, where the slope of a color histogram is set as an input value, or color image segmentation methods are applied [5][6][7].

2. Studies of Road Sign Recognition

In the area of road sign text recognition, recent studies have focused on text recognition in an external environment[8], which involves detecting signs in photos taken outside and recognizing whole words using a hidden Markov model-based application. The recognition rates for words, numbers, and symbols are 67.21%, 64.13%, and 80%, respectively, using this method.

The application of local binarization to text recognition in natural scene image is difficult for road signs with irregular text placement because users have to select the text area[9][10] while the processing speed is also low.

III. Image Processing before text Recognition

1. The Method of Image Processing for Text Recognition

Characters are extracted using an existing pattern recognition method. Hangul, the Korean alphabet, comprises initial-medial-final sounds, which means that it is more difficult to recognize than the English alphabet. When pattern recognition is applied to natural scene images in Hangul, the recognition rate is 60%, so an alternative method is needed that augments the recognition rate. In natural scene images, identical illumination conditions can be formed on the entire road sign surface or partially different conditions (the sun on the right or the left) may be generated depending on the external light environment. Cases 2 and 3 in <Table 1> have the same illumination conditions whereas Case 1 has different conditions. In the Cases 2 and 3, it is possible to set the average chromaticity value as the threshold value, whereas some of the characters cannot be extracted by defining the average chromaticity in Case 1. If variable conditions are present on the same sign, the chromaticity is distributed in no more than two separate areas, so it is necessary to divide an image into several sections, before merging the sections. When the chromaticity value reaches its upward inflection point, which is set as the threshold value, the part that contains the features of characters can be extracted from each binary image. Thus, to overcome the problem of Case 1, we introduce a method that specifies multiple threshold values within an image, before extracting the characters by segmenting the image and recombining the segments. In this case, the chromaticity values are expressed as polynomials, so the luminance of images generated by natural light rather than artificial illumination tends to have a certain distribution.

2. Color Band Collection for Suitable Value Application

During the extraction of attribute data from road sign destination names, such as directions and route information, preprocessing to increase the extraction rate is included only if people can see and read the data, where it is excluded if this is not possible. Computer vision only applies to the range of images that can be recognized by people. If this is not the case, a manual image data acquisition method is required to acquire recognizable images. Natural images contain a diverse range of color values, depending on the light intensity. <Table 1> shows the results of an analysis based on the colored parts of the range and the mean for a road sign in various environments. These results indicate that the areas containing characters cannot be extracted using the RGB color system alone. If a color system is used to extract white character areas from a green background, as shown in <Figure 2>, and an intersection occurs in the natural image in an area other than the original areas containing each color, then the color values in this area cannot be used to distinguish the area with characters. The same concept also applies to a blue background. A comparison of the intensity values for each RGB band in high-quality images among road sign images acquired in the field shows that there is a wide gap between the maximum and minimum values of the R band. This is because road signs are usually green or blue. In the present study, therefore, the R band was used for image preprocessing.

3. Image Division Method

Division of road sign images is applied if partial recognition is difficult because of the gradation that occurs when applying a global threshold, e.g., Case 1 in <Table 1>. This occurs when the intensity value of the R band is not bimodal. The bimodal status is determined by fitting a polynomial curve to the R band’s intensity value.

1) Deciding the sion of Division Number

The number of divisions in a road sign image is 1, if the R band intensity is bimodal. If not, multiple divisions are required. To determine the number of divisions, a polynomial expression is obtained by polynomial curve fitting using Eq. (1), where the degree is n.

Eq. (1) is used to derive the minimum value in an intensity graph based on the number of inflection points of the minimum value. A common bimodal case is a quartic polynomial. If this is not the case, the road signs are divided. The number of divisions is determined using Eq. (2).

2) Deciding the Division Direction

A road sign in a natural image contains gradation in a fixed direction because of the sun. Irregular gradation caused by other factors, such as artificial lighting and shadows, is not considered in the present study. With respect to the brightness of a road sign, gradation occurs in either a vertical or horizontal direction, and the division direction can be determined based on this direction. Thus, as shown in <Figure 3>, the division direction can be determined by dividing a road sign into four pieces and comparing the mean average intensities of these quadrants, which can be achieved using Eq. (3).

3) Application to Road Sign Images

MATLAB was used for the analysis and division of a natural image using various examples. Not all of the character areas could be extracted when global binarization was conducted using a single threshold. Thus, a single character area was extracted in T1, 1.2 areas in T2, 0.7 areas in T3, and 0.2 areas in T4. The corresponding threshold values are 0.15, 0.25, 0.35, and 0.45, respectively.

<Table 2> shows an image of Case 1 from <Table 1> after the application of image division and merging methods. In the case of road signs, a small image division with all of the chromaticity distributions applied is not required and the specification of no more than four threshold values allows the inclusion of all parts.

The use of this method does not guarantee 100% extraction of the character parts. Road signs contain place names, including current city names or brief city explanations such as city hall, so it is possible to compile a place name dictionary for a vicinity search. Thus, the correct name can be found for a word that is incorrectly or partially extracted by comparing the query with the results of the dictionary search. A regression analysis of chromaticity produces a polynomial with up to five terms, and this is followed by division of the part around the inflection point, which is determined by the differential as a threshold value. It is then possible to combine the texts extracted from the binary image that correspond to each merged image, as shown in <Table 2>. And also as shown in <Table 3>, applied several road sign images. Thus, using Eq. (1) and (2), determine ‘a’, ‘b’ values, and dividing directions of vertical or horizontal. After that, image processing will be done by multi-threshold. At the results, larger texts area were found in images, the accuracy of text recognition was improved.

V. Summary and Discussion

RReP, which comprises GPS, IMU, INS, DMI, and lasers, facilitates the automatic recognition of road signs and collects data on the shape, type, and pole location of road signs. RReP is expected to simplify the process of producing road sign registers by extracting basic data, including the sign shape and written content, as well as by automating database construction. The use of CDMA also allows a real- time approach to be applied to the road sign system. Thus, RReP is likely to address problems such as the unrealistic state of databases during the processing, storage, management, and renewal of data.

Image division and merging, which were applied in this study, produce superior results compared with local binarization method in terms of speed. Local binarization divides many areas directly before pixel processing, which means that this process is slower than simple image division based on the color intensity, which is used by our proposed method. Speed is important because it increases the efficiency of RReP, which demands the rapid processing of field images. Image preprocessing can be applied to the recognition of Korean and English texts. Thus, it can be applied to existing text recognition programs or via a learning process for recognizing English texts.

The development of software to facilitate the automatic extraction of high-volume road sign attribute data (data number, texts, pictorial symbols, and route numbers) using the road sign attribute data extraction module will help to ensure that this system assumes a leading position in the field of information recognition technology. If RReP is commercialized, it could be installed without any consideration of vehicle obsolescence or accidents because it is presented as a platform type application. RReP can be used immediately after sensor calibration, which means that the production costs will be significantly lower compared with existing MMS vehicles, while the production time will be shortened and data storage is also simplified.

In the future, the automation of field survey data analysis will help to devise steps and measures to facilitate real-time maintenance, formulate phased maintenance plans, establish a national-level road sign management system, reduce the related budget, and improve the management efficiency. In particular, the provision of up-to-date information on the nation’s infrastructure will increase the public’s confidence in national data and the usage of nationwide services.