You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
If I want to train to generate Japanese, do I need to change the entire words.txt file to Japanese as well? #6
changed the title [-]f I want to train to generate Japanese, do I need to change the entire words.txt file to Japanese as well?[/-][+]If I want to train to generate Japanese, do I need to change the entire words.txt file to Japanese as well?[/+]on Feb 6, 2025
Yes, the corpus in words.txt file should be changed to Japanese as well. And there are several other places should be adjusted for training on other languages:
In 3.1 Data Preparation, adjust the corpus in texts.txt and font files in font_dir to support certain languages.
In 3.3 Text Glyph Pretraining, adjust the corpus in words.txt, font files in font_path and text generation rules in dataloader to support certain languages. Besides, change the visual_encoder to certain language OCR model's vision backbone and modify the code and config for alignment.
In 3.4 Prior Guided Training, change the ocr model (ABINet) or re-train it to support linguistic supervison on certain language.
Regarding these cases, I can only provide some suggestions based on speculation. Firstly, verify whether the incorrect character is properly rendered by the font files during the pre-training phases, as the other character in val image seems to be fine. Besides, ensure that there is sufficient data available for training all the characters.
Many of the images in the training set are like this, with only half of the image present. At first, I thought it was due to image augmentation. Now that I've finished training, I modified the inference file to add mangaocr. I tried using it to generate images, but the generated characters only have half of them.
Should the training set include images that are only half present?
BTW, if you are using SRNet-Datagen to generate data, the problem where only half of the character image is displayed may be caused by insufficient padding during text rendering. You can try modifying the pad parameter in render_text_mask.py to see if it resolves the generation issue
I made some modifications to the CLIP training, adding embeddings for all Japanese fonts. The training process proceeded normally, but the generation of Japanese kanji is not very good. To be precise, it's not good on the test set, while it looks quite good on the training set.我对 CLIP 训练进行了一些修改，为所有日语字体添加了嵌入。训练过程进行得很正常，但日语汉字的生成不是很好。准确地说，它在测试集上并不好，而在训练集上看起来相当不错。
My training results are not very good. Could you please tell me how to make some modifications to CLIP training and how to add embeddings for all fonts.
I made some modifications to the CLIP training, adding embeddings for all Japanese fonts. The training process proceeded normally, but the generation of Japanese kanji is not very good. To be precise, it's not good on the test set, while it looks quite good on the training set.
I would like to ask how you can achieve the replacement of the Japanese OCR model?
I made some modifications to the CLIP training, adding embeddings for all Japanese fonts. The training process proceeded normally, but the generation of Japanese kanji is not very good. To be precise, it's not good on the test set, while it looks quite good on the training set.我对 CLIP 训练进行了一些修改，为所有日语字体添加了嵌入。训练过程进行得很正常，但日语汉字的生成不是很好。准确地说，它在测试集上并不好，而在训练集上看起来相当不错。
My training results are not very good. Could you please tell me how to make some modifications to CLIP training and how to add embeddings for all fonts.
I would like to ask if you also want to generate other languages? I didn't quite understand the conversation between the questioner and the author. The questioner wants to generate Japanese, but ABInet cannot recognize Japanese. Now I would like to ask if you know how to replace the OCR model that can recognize other languages? Hope to get your answer, thank you！
the training process uses a pretrained OCR model, ABINet, which is only trained on English.
Hello, I have the same problem as you. I want to generate Chinese, but ABINet can only recognize English. How did you solve this problem? Specifically, did you retrain ABInet? Or did you perform some operations through CLIP? I really appreciate your answer. Thank you.
Activity
XIONGPEILIN commentedon Jan 30, 2025
the training process uses a pretrained OCR model, ABINet, which is only trained on English.
[-]f I want to train to generate Japanese, do I need to change the entire words.txt file to Japanese as well?[/-][+]If I want to train to generate Japanese, do I need to change the entire words.txt file to Japanese as well?[/+]weichaozeng commentedon Feb 7, 2025
Yes, the corpus in words.txt file should be changed to Japanese as well. And there are several other places should be adjusted for training on other languages:
XIONGPEILIN commentedon Feb 8, 2025
thanks
XIONGPEILIN commentedon Feb 15, 2025
preglyph/modules.py
self.character = string.printable[:-6]
self.num_cls = len(self.character) + 1
This should also be adapted to Japanese.
weichaozeng commentedon Mar 16, 2025
Regarding these cases, I can only provide some suggestions based on speculation. Firstly, verify whether the incorrect character is properly rendered by the font files during the pre-training phases, as the other character in val image seems to be fine. Besides, ensure that there is sufficient data available for training all the characters.
XIONGPEILIN commentedon Mar 27, 2025
Should the training set include images that are only half present?
weichaozeng commentedon Mar 27, 2025
No, the training set requires intact and legible text images to ensure accurate model training.
weichaozeng commentedon Mar 30, 2025
BTW, if you are using SRNet-Datagen to generate data, the problem where only half of the character image is displayed may be caused by insufficient padding during text rendering. You can try modifying the pad parameter in render_text_mask.py to see if it resolves the generation issue
ManyDragons commentedon Apr 2, 2025
May I ask if words.txt and texts.txt can be used interchangeably? In other words, do the two use the same txt file
weichaozeng commentedon Apr 6, 2025
They dont need to be the same txt file, but they can be used interchangeably (as long as they exclude the illegal character).
ManyDragons commentedon Apr 14, 2025
Could you please confirm if the charset file referenced in the configuration file should also be replaced? Thank you!
ManyDragons commentedon Apr 27, 2025
My training results are not very good. Could you please tell me how to make some modifications to CLIP training and how to add embeddings for all fonts.
HHanTheKing commentedon Jul 1, 2025
I would like to ask how you can achieve the replacement of the Japanese OCR model?
HHanTheKing commentedon Jul 1, 2025
I would like to ask if you also want to generate other languages? I didn't quite understand the conversation between the questioner and the author. The questioner wants to generate Japanese, but ABInet cannot recognize Japanese. Now I would like to ask if you know how to replace the OCR model that can recognize other languages? Hope to get your answer, thank you！
HHanTheKing commentedon Jul 1, 2025
Hello, I have the same problem as you. I want to generate Chinese, but ABINet can only recognize English. How did you solve this problem? Specifically, did you retrain ABInet? Or did you perform some operations through CLIP? I really appreciate your answer. Thank you.
XIONGPEILIN commentedon Jul 2, 2025
I've modified CLIP's embedding layer to accommodate Japanese text input, and replaced ABINet with MangaOCR.