1.1.1 Chinese Writing Systems
1 Introduction
1.1CJK Scripts
Computers were developed mainly to support Western languages. However, with the rapid adoption of computing in non-western countries, certain issues began to arise. One major issue is how to deal with the Chinese, Japanese, and Korean writing systems. How could an input system designed for 52 phonetic symbols deal with literally thousands of ideographic symbols?
Unlike those of Western languages, CJK languages contain both syllable and ideographic characters. Some of these writing systems, such as Japanese hiragana and katakana, are linear. However, other systems, such as Japanese kanji are made up of two dimensional block characters. A more thorough description of block characters appears in section 1.1.4 “Block Characters”. A description of the main writing systems of China, Korea, and Japan, as well as some brief historical background, appears below.
1.1.1 Chinese Writing Systems
1.1.1.1Lading Zimu (Latin Characters)
These characters are simply the set of latin characters used in the Chinese language. They comprise the uppercase letters, lowercase letters, and numerals zero through nine. These are used in places such as mathematical formulas, charts, or transliteration. The most frequently used techniques for transliteration are Pinyin, used primarily in China, and Wade-Giles, used primarily in Taiwan.
1.1.1.2Zhuyin
Zhuyin, also commonly referred to as bopomofo, is a system used to transform Chinese characters into its simpler component elements based on the pronounciation. This system is nearly identical in China and Taiwan.
1.1.1.3Hanzi
Hanzi is the traditional system of Chinese characters. As of 1994, there were approximately 85,000 Hanzi in use in China.[1] In China, simplified versions of many character have come into common use. Taiwan, however, continues to use the traditional, more complex version of the characters.
1.1.2 Korean Writing Systems
1.1.2.1 Hangul
The Hangul writing system is a rather unique writing system made up of symbols formed of individual letters. These individual letters are called jamo. These hangul characters are used to create the words when writing in Korean.
1.1.2.2 Hanja
Chinese characters that are used in Korean are called hanja (same char as Japanese“kanji”). These were borrowed from China on the characters’ way to Japan. In the past, hanja were used quite frequently. However, in modern Korea, most writing is done in Hangul.
1.1.3 Japanese Writing Systems
1.1.3.1 Hiragana
Over the years, within the writing system imported from china, a subset started to form. This set of characters was a greatly simplified version of some commonly used Chinese characters. Also, in a break from the original style of the Chinese writing system, these extremely simplified characters were used as a set of the possible syllables that made up the Japanese language. This was a marked difference from the ideographical system of writing that had originally been imported from the Chinese mainland.
This is not to say that the Japanese writing system stopped using ideographical characters. Rather, the writing system consists of a mix of furigana and kanji (Chinese characters). For instance, これはひらがなと漢字があります. “This has hiragana and Kanji” is a sentence that uses both scripts. This leads to a very complicated system, where a given character can have multiple possible phonetic readings. For example, hi, ni, nichi are all readings of the character 日.
1.1.3.2 Katakana
Katakana is similar to hiragana in some ways. For example, katakana, like hiragana, is a syllabary. Also there are corresponding characters for each character found in the hiragana syllabary. Many of the katakana are extremely similar to their corresponding hiragana character. In fact, most of these similar characters are derived from the same Kanji. The most common uses of katakana are for foreign loan words, names of those who do not have a Chinese character name, onomatopoeic words, and to add emphasis.
For example, “Scott” is written as スコットrather than そこっと. Also, the word ______is an example of onomatopoeia. Japanese words are sometimes written in katakana to show emphasis. An example might be イヤダ！In Japanese, many foreign loan words that aren’t of Chinese origin are from English. Words like バスケットボール(Basketball) or テレビ (Televeision) are common examples of this phenomenon.
1.1.3.3 Kanji
Chinese characters, or 漢字, were borrowed extensively up until 1300 by Japan.[2] Most of these characters came into Japan over three distinct periods, in the form of kanji compounds. Compounds are multi-syllable words. Often times, when compounds borrowed from different eras contained the same symbol, the symbol would be pronounced differently in the different compounds.
Another character type is the set of 国字(こくじ), which is the set of Chinese-style characters created in Japan. Examples include characters such as 働(はたらく) and 枠(わく). This class of characters came about to create needed new words at a time when Japan was in isolation, and therefore unable to borrow a character.
Something about the 音読み おんよみ and 訓読みくんよみ
The characters borrowed from Chinese have readings that can be classified as one of two types. The first possible reading is the kunyomi, which is the reading that comes from the original Japanese reading. These come from before the first wide scale integration of Chinese characters. The second reading is the onyomi. This is the reading that came from the Chinese reading. It is possible for a character to have more than one onyomi.
1.1.3.4 Latin(European)
A system of romanization exists for the Japanese writing system. It uses the standard western alphabet to transliterate the sounds of words written in katakana, hiragana, and kanji. One of the more widely used systems is called the Hepburn system.
か= ka
カ= ka
家= ka
Figure XXX
Figure XXX shows an example of hiragana, katakana, and kanji characters, all of which can be transliterated to “ka”.
In modern japan, there is also frequent use of the latin alphabet for purposes other than transliterations. Sometimes this is for multinational corporations, where the company wishes to do business overseas. The company in this situation will usually write its name in Latin characters. Some examples are Sony and Sanyo. Popular culture also frequently uses latin script, and English words in advertising and other media.
1.1.4 Block Characters
The Korean system of [Korean writing system-not Chinese characters], as well as the use of Chinese characters in Japanese, Korean, and Chinese, are all examples of block characters. For an example of what is meant by “block characters”, consider the following example. The Japanese character 口by itself is pronounced くち, and means mouth. This character is also a component of a number of other 漢字. For example, 言, 味, 語,裕,邑,右, to name just a few. This ability of one character to appear in multiple positions in another character makes it a block character (two dimensional). It is not only stand alone, but also components called “radicals” that can appear in various places in kanji.
In Korean, a system with more direct block structure is in place. More information about how these are composed, and what makes them block characters.
1.1.5 Problems with Existing Techniques
Despite the two dimensional characteristics found in these block character writing systems, there is currently no viable system in place to take advantage of the 2 dimensional nature of the characters. Use of a standard keyboard requires that characters be input serially. However, the serialization schemes used, while efficient for typing, offer little if any insight into the composition into the characters. Instead, they are simply hashing functions.
In order to exploit the information built into the two-dimensional form of the characters, another serialization technique is needed. Section 1.2 describes my serialization technique. In regular typing, an extended serialization technique that reveals structure would not be especially efficient. This serialization technique is not intended to replace the hashing methods in those instances. Instead, it is meant to provide an alternate means of searching for text in a document, or determining which character to use, without requiring the ability to form the entire character desired.
1.2 Approach: Extended Regular Expressions
The need then, is to find a technique to accurately describe these three-dimensional writing systems in what is an inherently two-dimensional system. Also, once this serialization scheme is in place, it seems only logical to extend it to allow text searches on Asian languages the same amount of freedom available on western language text. The serialization of the characters has allowed them to now be treated as non-atomic units. This is the key to allowing this new approach.
[1]Page 58, CJKV
[2] P.59 cjkv