Repository navigation
The issues in scene text editing in a different language #14
Description
Hello! We are attempting to perform AR translation from a different language font to English. However, we've encountered the following issue.
- adding alphabets instead of representing the input characters as they are
- the input characters appear in two lines
Do you have any insights into possible reasons for this?
Activity
Question406 commentedon Sep 11, 2023
hello, our model may be less effective in this abnormal aspect ratio mask since we are training on ground-truth English character images with more narrow masks.
I think you can try several things to improve the result: 1. Try more random seed and guidance scales. 2. First in-paint the masked part with an inpainting model and then generate the target characters with a less wide mask.
Neks11 commentedon Mar 25, 2024
Hello,
Thanks for sharing your work.
Can results be improved for wider mask, curved mask by fine tuning the model.
Did you try the same?
Also model is not giving good results for exiting Text font style transfer with curved, wider mask.
Question406 commentedon Apr 15, 2024
@Neks11 Hi, the current synthetic data generation pipeline in our work does not generate many authentic curved texts, which is limited by the rendering engine. I think you could try more advanced data augmentation. Another direction worth trying is the Anytext(https://github.com/tyxsspa/anytext) and TextDiffuser(https://jingyechen.github.io/textdiffuser/), which incorporates both text masks and direct glyph-level guidance with character scene graphs.