Tangut characters were created by the Tangut of the Western Xia (Xi Xia) Dynasty in ancient China and are over 1000 years old. In deep-learning-based recognition studies on Tangut characters, the lack of category-complete datasets has been problematic. Data augmentation cannot augment the character categories of unknown styles, whereas the use of image generation can effectively solve the problem. In this study, we consider the generation of antique book calligraphy styles of Tangut characters as a problem of learning to map from existing printed styles to personalized antique book calligraphy styles. We present M-ResNet, a multi-scale feature extraction residual unit, and Tangut-CycleGAN, a model for generation Tangut characters that combine M-ResNet and a cycle-consistent adversarial network (CycleGAN). This method uses unpaired data to generate Tangut character images in the calligraphy style of ancient books. To enhance the response of the model to significant channels, a squeezing-and-excitation (SE) module is introduced based on Tangut-CycleGAN to design the Tangut-CycleGAN+SE method for generating images of Tangut characters. This method is not only suitable for Tangut character image generation, but also can effectively generate calligraphy with aesthetic value. In addition, we propose an overall quality discrepancy evaluation metric, FA (Fréchet inception distance + Accuracy), to evaluate the quality of character image generation, which combines style discrepancy and content accuracy metrics.
XuS., JinT., JiangH. and LauF.C., Automatic generation of personal chinese handwriting by capturing the characteristics of personal handwriting, in: Twenty-First IAAI Conference, 2009, pp. 191–196.
2.
XuS., LauF.C., CheungW.K. and PanY., Automatic generation of artistic chinese calligraphy, IEEE Intelligent Systems20(3) (2005), 32–39.
3.
LiuP., XuS. and LinS., Automatic generation of personalized chinese handwriting characters, in: 2012 Fourth International Conference on Digital Home, IEEE, 2012, pp. 109–116.
IsolaP., ZhuJ.Y., ZhouT. and EfrosA.A., Image-to-image translation with conditional adversarial networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1125–1134.
6.
GoodfellowI., Pouget-AbadieJ., MirzaM., XuB., Warde-FarleyD., OzairS., CourvilleA. and BengioY., Generative adversarial nets, Advances in Neural Information Processing Systems27(2014), 1–9.
PathakD., KrahenbuhlP., DonahueJ. and DarrellT., EfrosA.A., Context encoders: Feature learning by inpainting, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2536–2544.
9.
IizukaS., Simo-SerraE. and IshikawaH., Globally and locally consistent image completion, ACM Transactions on Graphics(ToG)36(4) (2017), 1–14.
10.
DentonE.L., ChintalaS., SzlamA. and FergusR., Deep generative image models using a laplacian pyramid of adversarial networks, Advances in Neural Information Processing Systems28 (2015), 1486–1494.
11.
RadfordA., MetzL. and ChintalaS., Unsupervised representation learning with deep convolutional generative adversarial networks, arXiv preprint arXiv:1511.06434 (2015).
12.
ZhuJ.Y., KrähenbühlP., ShechtmanE. and EfrosA.A., Generative visual manipulation on the natural image manifold, in: European Conference on Computer Vision, Springer, 2016, pp. 597–613.
13.
MathieuM.F., ZhaoJ.J., ZhaoJ., RameshA., SprechmannP. and LeCunY., Disentangling factors of variation in deep representation using adversarial training, Advances in Neural Information Processing Systems29 (2016), 5047–5055.
14.
ZhuJ.Y., ParkT., IsolaP. and EfrosA.A., Unpaired image-to-image translation using cycle-consistent adversarial networks, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2223–2232.
15.
OordA.V.D., DielemanS., ZenH., SimonyanK., VinyalsO., GravesA., KalchbrennerN., SeniorA. and KavukcuogluK., Wavenet: A generative model for raw audio, arXiv preprint arXiv:1609.03499 (2016).
16.
JotyS., NakovP., MàrquezL. and JaradatI., Cross-language learning with adversarial neural networks: Application to community question answering, arXiv preprint arXiv:1706.06749 (2017).
MaoH. and PanW., A style transformation method for printed chinese characters, in: International Conference on Image and Graphics, Springer, 2015, pp. 494–503.
19.
ChangJ. and GuY., Chinese typography transfer, arXiv preprint arXiv:1707.04904 (2017).
20.
HeK., ZhangX., RenS. and SunJ., Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.
21.
EfrosA.A. and LeungT.K., Texture synthesis by non-parametric sampling, in: Proceedings of the Seventh IEEE International Conference on Computer Vision2 (1999), 1033–1038.
22.
LongJ., ShelhamerE. and DarrellT., Fully convolutional networks for semantic segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 3431–3440.
23.
KaracanL., AkataZ., ErdemA. and ErdemE., Learning to generate images of outdoor scenes from attributes and semantic layouts, arXiv preprint arXiv:1612.00215 (2016).
24.
SangkloyP., LuJ., FangC., YuF. and HaysJ., Scribbler: Controlling deep image synthesis with sketch and color, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 5400–5409.
IoffeS. and SzegedyC., Batch normalization: Accelerating deep network training by reducing internal covariate shift, in: International Conference on Machine Learning, PMLR, 2015, pp. 448–456.
27.
GlorotX., BordesA. and BengioY., Deep sparse rectifier neural networks, in: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, JMLR Workshop and Conference Proceedings, 2011, pp. 315–323.
28.
HuJ., ShenL. and SunG., Squeeze-and-excitation networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7132–7141.
29.
HowardA., SandlerM., ChenB., WangW., ChenL.C., TanM. and ChuV.G., Searching for mobilenetv3, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 1314–1324.
30.
BodapatiJ.D., ShareefS.N., NaralasettiV. and MundukurN.B., Msenet: multi-modal squeeze-and-excitation network for brain tumorseverity prediction, International Journal of Pattern Recognition and Artificial Intelligence35(07) (2021), 2157005.
31.
RoyS.K., DubeyS.R., ChatterjeeS. and ChaudhuriB.B., Fusenet: Fused squeeze-and-excitation network for spectral-spatial hyperspectral image classification, IET Image Processing14(8) (2020), 1653–1661.
32.
MaJ., CaoY., MaZ., WeiL. and HaoC., End-to-end tangut character database building and recognition method, IET Image Processing (2022) 1–14.
33.
ChangB., ZhangQ., PanS. and MengL., Generating hand-written chinese characters using cyclegan, in: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE, 2018, pp. 199–207.
34.
HeuselM., RamsauerH., UnterthinerT., NesslerB. and HochreiterS., Gans trained by a two time-scale update rule converge toa local nash equilibrium, Advances in Neural Information Processing Systems30 (2017), 6626–6637.
35.
XiaoX., JinL., YangY., YangW., SunJ. and ChangT., Buildingfast and compact convolutional neural networks for offlinehandwritten chinese character recognition, Pattern Recognition72 (2017), 72–81.
36.
ZhongZ., JinL. and XieZ., High performance offline hand-written chinese character recognition using googlenet and directional feature maps, in: 2015 13th International Conference on Document Analysis and Recognition (ICDAR), IEEE, 2015, pp. 846–850.
37.
KingmaD.P. and BaJ., Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014).
38.
GatysL.A., EckerA.S. and BethgeM., A neural algorithm of artistic style, arXiv preprint arXiv:1508.06576 (2015).
39.
HouQ., ZhouD. and FengJ., Coordinate attention for efficient mobile network design, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 13713–13722.
40.
HowardA., ZhmoginovA., ChenL.C., SandlerM. and ZhuM., Inverted residuals and linear bottlenecks: Mobile networks for classification, detection and segmentation (2018).
41.
ZhouD., HouQ., ChenY., FengJ. and YanS., Rethinking bottleneck structure for efficient mobile network design, in: European Conference on Computer Vision, Springer, 2020, pp. 680–697.
42.
TanM. and LeQ., Efficientnet: Rethinking model scaling for convolutional neural networks, in: International conference on machine learning, PMLR, 2019, pp. 6105–6114.
