The Japanese writing system is well known for its complexity. To even write the most basic Japanese sentences, knowledge of three scripts is vital: Hiragana, Katakana (two sets of syllabic alphabets), and Kanji (thousands of Chinese characters). Each of these has specific roles in a sentence, so it is essential to master all three scripts to read and write in Japanese. This complex system serves as a barrier, making the land and its culture harder to access from outside.
But there is another, less well-known, barrier inside Japan. On account of several writing reforms that took place in the second half of the 19th century, most Japanese natives today – including me – are unable to read books produced just 150 years ago. This means that the country suffers from the difficulties of transmitting its own past knowledge to modern society. In Japanese libraries and households, there are millions of books and billions of documents that have been waiting to be read one day.
Until the mid-19th century, the most common book making technique in Japan had been the woodblock printing, which is a faithful reproduction of a manuscript.
Handwritten books were also valued and circulated throughout history. Source: CODH (Collection: National Institute of Japanese Literature)
Kuzushiji or obstacles to reading pre-modern Japanese texts
There are two major obstacles to reading pre-modern Japanese texts. The first obstacle is that pre-modern Japanese texts are printed (or written) in an extreme form of cursive. In this style, identifying the original characters is a challenge. Not only do they look different from today’s Hiragana, but they also wildly connect to the preceding and the following characters. It is even hard to recognize the beginnings and ends of each character.
The second obstacle is that, unlike today’s system of strictly one-character-for-one sound, there used to be many ways of writing one sound in Hiragana. That means you need to remember at least three, sometimes more than ten variations of the Hiragana character. Some alternate Hiragana shapes look identical to Kanji characters, too. The only way to recognize them correctly is to determine them from the context.
This rather particular form of writing is known as “Kuzushiji”. Kuzushiji had been in fact in use for over a thousand years, but nowadays only trained experts can read it.
This is why I was extremely impressed when I read about Dr. Tarin Clanuwat – a young scholar from Thailand – who developed an AI-based character recognition system, which magically transcribes pre-modern Japanese characters into contemporary Japanese characters.
Fascination for medieval Japan
Tarin has been fascinated by Japanese culture since her childhood. As a teenager, she taught herself Japanese but her interest at this stage was still Japanese culture in general. As a university student in Thailand – she majored in Japanese, of course – she encountered her true passion: medieval Japanese culture. She was captivated by the manga version of “The Tale of Genji” – a masterpiece of Japanese literature written in the 11th century. In 2008, Tarin moved to Japan to take a postgraduate course in Japanese medieval literature. The major problem she encountered there, however, was: Kuzushiji!
One of many The Tale of Genji book scrolls. Source: CODH (Collection: National Institute of Japanese Literature)
A spread from the manga version of The Tale of Genji, which is also considered as one of many masterpieces of Japanese comics. Source: https://kc.kodansha.co.jp/product?item=0000359115
Being able to decipher Kuzushiji is an essential skill in her subject, yet gaining proficiency in it is a challenging task. Having once failed an exam on Kuzushiji, she thought “Well, if I could write it myself, I should be able to read it!”. Tarin signed up for calligraphy lessons and eventually achieved not only her initial goal (of being able to read Kuzushiji) – but became an excellent calligrapher as well.
AI object-detection for Kuzushiji recognition
Even though she was armed with specialised training, reading Kuzushiji remained an extremely laborious task. Gradually, Tarin contemplated the idea of automating it.
Because of the idiosyncratic issues of Kuzushiji – the complexity of the Kuzushiji and the obscure layout of Japanese books – the conventional optical character recognition (OCR) was incapable of providing reliable results.
So Tarin had the idea of utilizing an object-detection algorithm, which is used in medical research to analyse images of cells. Unlike conventional OCR, which first configures the layout, then moves on to recognize individual characters, object detection enables the system to simultaneously identify multiple items and levels on a page. The organic and complex way of Kuzushiji writing apparently reminded her of cell images – and this unconventional approach turned out to be a great success. It provides significantly more accurate results than conventional OCR methods.
Using an object-detection algorithm had the advantage that it is a well-researched area, so there were many existing models to try out. Tarin and her team assessed the advantages and disadvantages of different approaches, in order to develop the most appropriate system for Kuzushiji recognition.
After months of researching, making prototypes and testing, in 2019, Tarin and her team at the Center for Open Data in the Humanities published the first version of KroNet, a web app for professionals (mainly used by scholars). As one would expect, the KroNet system gained extensive attention inside and outside of Japan.
But she didn’t stop there. It was her passion to make historical materials written in Kuzushiji accessible to the general public. In 2021, Tarin and her team went on to publish a mobile app called Miwo, which can be downloaded and used by anyone.
Of course, I downloaded Miwo and used it – it is very easy to use and really fun! Transcribing the text on the spot is really powerful, as you can find Kuzushiji on so many historical materials – maps, ads and artworks. Until recently, I could only see Kuzushiji as some sort of decoration, but not any more with Miwo! The app will certainly contribute so much to building a bridge between the past, present and future of Japanese culture.
Lastly but importantly: the dataset – a great number of printed and handwritten characters – that was used to train the AI systems of KroNet and Miwo, is publicly available. It can be rich resource for type designers or design researchers to see historical forms of Japanese characters.
The dataset of historical Japanese characters, which was used to train the AI of KroNet (http://codh.rois.ac.jp/kuronet/) and Miwo (http://codh.rois.ac.jp/miwo/) is publicly available for humans as well as for machines. The page above shows the variations of character E (え).
The page above shows the variations of character Mo (も) .
Clanuwat, T., Lamb, A., and Kitamoto, K. (2020). “KuroNet: Regularized Residual U‐Nets for End‐to‐End Kuzushiji Character Recognition”
Clanuwat, T. (2020)『くずし字認識の進化とサービス化の展開』
Clanuwat, T., Lamb, A., and Kitamoto, K. (2019). “KuroNet: Pre-Modern Japanese Kuzushiji Character Recognition with Deep Learning”
Clanuwat, T., Lamb, A., and Kitamoto, K. (2019) “A Human-Inspired Recognition System for Pre-Modern Japanese Historical Documents”
Reading Edo: Data-driven Approaches for Japan Studies
The 95th International ARC Seminar
Tarn Clanuwat, MIWO: Kuzushiji recognition smartphone application with AI
CDDP Video Series: Kuronet Kuzushiji Recognition--IIIF Curation Viewer
Tarin’s social media