Developing a Web Application for Chinese Calligraphy Learners using Convolutional Neural Network and Scale Invariant Feature Transform

Chinese Calligraphy is an ancient art form with practical benefits such as improving elders' cognition and helping diagnose Parkinson's disease. The characters in Chinese calligraphy are composed of strokes of an inked brush. Learning Chinese calligraphy requires a lot of time and effort. The number of strokes in Chinese characters can vary from one to more than thirty. There are more than 2,500 Chinese characters that are used daily. Previous research on Chinese calligraphy style recognition or character recognition used small-sized datasets of up to 300 characters. There is a need for more research on AI models which combine style and character recognition on a much larger dataset. There is also no application of such models to help learners of Chinese calligraphy compare their works with those created by skillful calligraphers. This research proposed two AI models using the VGG(Visual Geometry Group)-net architecture for style recognition and character recognition for more than 900 Chinese characters in five calligraphic styles.
Furthermore, we proposed using the techniques of data augmentation and dropout to improve the accuracy. In the proposed models, style recognition achieved up to 88.3% accuracy, while character recognition achieved 89% accuracy. We also combined the model with Scale Invariant Feature Transform (SIFT) to create a web application for calligraphy learners. The application can compare the learner's work with that of skillful calligraphers and provide a similarity score. Five university students tested the application, and some qualitative data were collected for future enhancement.