Skip to content

【Hackathon 5th】fastdeploy版的sd15模型添加gradio界面 - #311

Merged
JunnYu merged 13 commits into
PaddlePaddle:developfrom
sanbuphy:develop
Dec 27, 2023
Merged

【Hackathon 5th】fastdeploy版的sd15模型添加gradio界面#311
JunnYu merged 13 commits into
PaddlePaddle:developfrom
sanbuphy:develop

Conversation

@sanbuphy

@sanbuphy sanbuphy commented Nov 18, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

支持文生图、图生图、inpaint_legacy、inpaint、controlnet+canny的推理
#256

aistudio 项目还在施工中。

@JunnYu
JunnYu self-requested a review November 20, 2023 02:49
JunnYu
JunnYu reviewed Nov 20, 2023
View reviewed changes
Comment thread ppdiffusers/deploy/gradio_demo.py Resolved
JunnYu
JunnYu reviewed Nov 20, 2023
View reviewed changes
Comment thread ppdiffusers/deploy/gradio_demo.py Resolved
Comment thread ppdiffusers/deploy/gradio_demo.py Resolved
@sanbuphy
sanbuphy requested a review from JunnYu November 24, 2023 03:39
@JunnYu

JunnYu commented Nov 24, 2023

Copy link
Copy Markdown
Member

你好，需要添加更多的参数到gradio界面，比如negative_prompt，guidance scale

@sanbuphy

sanbuphy commented Nov 24, 2023

Copy link
Copy Markdown
Contributor Author

你好，需要添加更多的参数到gradio界面，比如negative_prompt，guidance scale

需要全部吗？因为 pipe那边还满多的，还是说常见的即可呢

@JunnYu

JunnYu commented Nov 24, 2023 •
edited
Loading

Copy link
Copy Markdown
Member
  • 需要一些常见的这种参数。
  • 当前我测试了一下，有个可以跑通推理的paddle和fd及ppdiffusers环境，之后我提供一下版本，你可以继续这部分开发了
  • 我看了一下inpaint的界面，可能需要自动涂抹mask的工具，无需手动上传

@sanbuphy

sanbuphy commented Nov 24, 2023

Copy link
Copy Markdown
Contributor Author
  • 需要一些常见的这种参数。
  • 当前我测试了一下，有个可以跑通推理的paddle和fd及ppdiffusers环境，之后我提供一下版本，你可以继续这部分开发了
  • 我看了一下inpaint的界面，可能需要自动涂抹mask的工具，无需手动上传

1.2 : 收到
3 ：我看看gradio 的组件是否支持这种形式

This was referenced Nov 25, 2023
Merged
Closed
@JunnYu

JunnYu commented Nov 28, 2023 •
edited
Loading

Copy link
Copy Markdown
Member

如果想要导出模型，请使用ppdiffusers==0.16.3进行导出。

推理的环境：

# aistudio的v100环境下
python -m pip install paddlepaddle-gpu==2.5.2 -i https://pypi.tuna.tsinghua.edu.cn/simple --user
pip install fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html --user
pip install https://paddlenlp.bj.bcebos.com/models/community/junnyu/fd_trt_model/ppdiffusers-0.19.3-py3-none-any.whl --user

适配文生图，图生图，inpaint legacy的模型

# 下载导出后的模型
# 这个是适配文生图，图生图，inpaint legacy的模型
wget https://paddlenlp.bj.bcebos.com/models/community/junnyu/fd_trt_model/runwayml_stable-diffusion-v1-5_fd.tar.gz && tar -zxvf runwayml_stable-diffusion-v1-5_fd.tar.gz

python infer.py --model_dir runwayml_stable-diffusion-v1-5_fd --task_name text2img
python infer.py --model_dir runwayml_stable-diffusion-v1-5_fd --task_name img2img
python infer.py --model_dir runwayml_stable-diffusion-v1-5_fd --task_name inpaint_legacy
image

适配inpaint的模型

wget https://paddlenlp.bj.bcebos.com/models/community/junnyu/fd_trt_model/stable-diffusion-v1-5-inpainting.tar.gz && tar -zxvf stable-diffusion-v1-5-inpainting.tar.gz
python infer.py --model_dir stable-diffusion-v1-5-inpainting --task_name inpaint
image

适配Controlnet的模型

# 使用的是controlnet目录下的infer
cd controlnet
# 下载
wget https://paddlenlp.bj.bcebos.com/models/community/junnyu/fd_trt_model/control_sd15_canny.tar.gz && tar -zxvf control_sd15_canny.tar.gz
python infer.py --model_dir control_sd15_canny --task_name text2img_control
image

@sanbuphy

sanbuphy commented Dec 2, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

@JunnYu 验证如下：(目前是普通的后端跑通但留了fd的接口）

普通pipe

image
image
image

inpaint

image

canny

image

JunnYu
JunnYu reviewed Dec 5, 2023
View reviewed changes
Comment thread ppdiffusers/deploy/gradio_demo.py Resolved
JunnYu
JunnYu reviewed Dec 5, 2023
View reviewed changes
Comment thread ppdiffusers/deploy/gradio_demo.py Resolved
JunnYu
JunnYu reviewed Dec 5, 2023
View reviewed changes
Comment thread ppdiffusers/deploy/gradio_demo.py Resolved
@JunnYu

JunnYu commented Dec 7, 2023

Copy link
Copy Markdown
Member

现在咋样了呢？这个gradio界面能跑成功了吗？

@sanbuphy

sanbuphy commented Dec 7, 2023

Copy link
Copy Markdown
Contributor Author

现在咋样了呢？这个gradio界面能跑成功了吗？

sorry这两三天加班都到九点多，我已经把模型拉到本地了，明晚上看看能不能再给新的一版

@sanbuphy
sanbuphy requested a review from JunnYu December 9, 2023 09:33
@sanbuphy

sanbuphy commented Dec 9, 2023

Copy link
Copy Markdown
Contributor Author

现在咋样了呢？这个gradio界面能跑成功了吗？

修正了大小 和图生图 inpaint问题：

image

image

@sanbuphy sanbuphy mentioned this pull request Dec 10, 2023
Closed
24 of 26 tasks
@sanbuphy

sanbuphy commented Dec 11, 2023

Copy link
Copy Markdown
Contributor Author

@JunnYu 麻烦研发大哥有空看看~

@shiyutang shiyutang changed the title fastdeploy版的sd15模型添加gradio界面 【Hackathon 5th】fastdeploy版的sd15模型添加gradio界面 Dec 13, 2023
JunnYu
JunnYu reviewed Dec 19, 2023
View reviewed changes
Comment thread ppdiffusers/deploy/gradio_demo.py Resolved
JunnYu
JunnYu reviewed Dec 19, 2023
View reviewed changes
Comment thread ppdiffusers/deploy/gradio_demo.py Resolved
@JunnYu

JunnYu commented Dec 19, 2023

Copy link
Copy Markdown
Member

我跑了一下，文生图、图生图、inpaint没有问题。
image

@JunnYu

JunnYu commented Dec 19, 2023

Copy link
Copy Markdown
Member

再添加一下对应的使用的readme，以及使用结果的截图。

@sanbuphy

sanbuphy commented Dec 19, 2023

Copy link
Copy Markdown
Contributor Author

再添加一下对应的使用的readme，以及使用结果的截图。

图片文件存在哪，可以新建asset文件夹吗

@JunnYu

JunnYu commented Dec 21, 2023

Copy link
Copy Markdown
Member

@sanbuphy 直接放到github这里，比如拖拽个图片进来
image

JunnYu
JunnYu previously approved these changes Dec 21, 2023
View reviewed changes

@JunnYu JunnYu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sanbuphy

sanbuphy commented Dec 21, 2023

Copy link
Copy Markdown
Contributor Author

@sanbuphy 直接放到github这里，比如拖拽个图片进来 image

明白，我写下readme

@JunnYu

JunnYu commented Dec 22, 2023

Copy link
Copy Markdown
Member

TODO内容：

  1. readme完善
  2. aistudio部署gradio项目
    LGTM

@sanbuphy
sanbuphy requested a review from JunnYu December 26, 2023 15:20
@sanbuphy

sanbuphy commented Dec 26, 2023

Copy link
Copy Markdown
Contributor Author

TODO内容：

  1. readme完善
  2. aistudio部署gradio项目
    LGTM

已部署成功，已添加readme

JunnYu
JunnYu reviewed Dec 27, 2023
View reviewed changes
Comment thread ppdiffusers/deploy/README.md Outdated

pip install fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html

pip install https://paddlenlp.bj.bcebos.com/models/community/junnyu/fd_trt_model/ppdiffusers-0.19.3-py3-none-any.whl

@JunnYu JunnYu Dec 27, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0.19.4 不用这个版本了

@JunnYu

JunnYu commented Dec 27, 2023

Copy link
Copy Markdown
Member

image
image
image

@JunnYu
JunnYu self-requested a review December 27, 2023 11:40
JunnYu
JunnYu approved these changes Dec 27, 2023
View reviewed changes

@JunnYu JunnYu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@JunnYu
JunnYu merged commit 9769ebe into PaddlePaddle:develop Dec 27, 2023
@sanbuphy sanbuphy mentioned this pull request Jan 19, 2024
Closed
18 of 24 tasks
@bigmover

bigmover commented Mar 6, 2024

Copy link
Copy Markdown

支持文生图、图生图、inpaint_legacy、inpaint、controlnet+canny的推理 #256

aistudio 项目还在施工中。

请问这边的文生图/图生图接口，会和diffusers相关接口保持同步吗？比如StableDiffusionControlNetInpaintPipeline， StableDiffusionControlNetImg2ImgPipeline 这种接口好像diffusers的参数他有时候会增加的

@sanbuphy

sanbuphy commented Mar 9, 2024

Copy link
Copy Markdown
Contributor Author

支持文生图、图生图、inpaint_legacy、inpaint、controlnet+canny的推理 #256
aistudio 项目还在施工中。

请问这边的文生图/图生图接口，会和diffusers相关接口保持同步吗？比如StableDiffusionControlNetInpaintPipeline， StableDiffusionControlNetImg2ImgPipeline 这种接口好像diffusers的参数他有时候会增加的

你好，比较困难，没有ci覆盖不知道什么时候坏了需要修理，只能固定一个版本

westfish pushed a commit to westfish/PaddleMIX that referenced this pull request Sep 25, 2024
@sanbuphy
【Hackathon 5th】fastdeploy版的sd15模型添加gradio界面 (PaddlePaddle#311)
1585822 
支持文生图、图生图、inpaint_legacy、inpaint、controlnet+canny的推理
PaddlePaddle#256

aistudio 项目还在施工中。
lyuwenyu added a commit that referenced this pull request Feb 20, 2025
@lyuwenyu
[wip] datacopilot analysis, filter and visualize (#1060)
0d362e3 
## 算子目录

- [1. 转换算子](#1-转换算子)
  - [1.1 llava转换算子](#11-llava转换算子)
    - [1.1.1 llava_convert](#111-llava_convert)
- [2. 过滤算子](#2-过滤算子)
  - [2.1 基础过滤算子](#21-基础过滤算子)
    - [2.1.1 valid_data_filter](#211-valid_data_filter)
- [2.1.1.1 image_compliance_operator](#2111-image_compliance_operator)
- [2.1.1.2
conversation_compliance_operator](#2112-conversation_compliance_operator)
  - [2.2 文本过滤算子](#22-文本过滤算子)
- [2.2.1 conversation_length_filter](#221-conversation_length_filter)
- [2.2.2 average_line_length_filter](#222-average_line_length_filter)
- [2.2.3 maximum_line_length_filter](#223-maximum_line_length_filter)
- [2.2.4
conversation_percentage_filter](#224-conversation_percentage_filter)
    - [2.2.5 token_num_filter](#225-token_num_filter)
    - [2.2.6 alphanumeric_ratio_filter](#226-alphanumeric_ratio_filter)
    - [2.2.7 stopwords_ratio_filter](#227-stopwords_ratio_filter)
    - [2.2.8 special_characters_filter](#228-special_characters_filter)
    - [2.2.9 language_id_filter](#229-language_id_filter)
    - [2.2.10 text_action_filter](#2210-text_action_filter)
- [2.2.11
text_entity_dependency_filter](#2211-text_entity_dependency_filter)
- [2.2.12
char_ngram_repetition_filter](#2212-char_ngram_repetition_filter)
- [2.2.13
word_ngram_repetition_filter](#2213-word_ngram_repetition_filter)
    - [2.2.14 conversation_hash_filter](#2214-conversation_hash_filter)
- [2.2.14.1
simhash_duplicate_operator](#22141-simhash_duplicate_operator)
- [2.2.14.2
minhash_duplicate_operator](#22142-minhash_duplicate_operator)
    - [2.2.15 llm_judge_filter](#2215-llm_judge_filter)
  - [2.3 图像过滤算子](#23-图像过滤算子)
    - [2.3.1 image_filesize_filter](#231-image_filesize_filter)
    - [2.3.2 image_ration_filter](#232-image_ration_filter)
    - [2.3.3 image_resolution_filter](#233-image_resolution_filter)
    - [2.3.4 image_hash_filter](#234-image_hash_filter)
  - [2.4 图文过滤算子](#24-图文过滤算子)
    - [2.4.1 image_clip_filter](#241-image_clip_filter)
- [3. 分析算子](#3-分析算子)
  - [3.1 基础分析算子](#31-基础分析算子)
    - [3.1.1 base_analysis_pipeline](#311-base_analysis_pipeline)
- [3.1.1.1 analyze_dataset_statistics](#3111-analyze_dataset_statistics)
- [3.1.1.2
analyze_language_distribution](#3112-analyze_language_distribution)
      - [3.1.1.3 analyze_image_paths](#3113-analyze_image_paths)
      - [3.1.1.4 analyze_data_anomalies](#3114-analyze_data_anomalies)
- [3.1.1.5
analyze_conversation_tokens](#3115-analyze_conversation_tokens)
  - [3.2 进阶分析算子](#32-进阶分析算子)
    - [3.2.1 description_analysis](#321-description_analysis)
    - [3.2.2 quality_analysis](#322-quality_analysis)
- [4. 可视化算子](#4-可视化算子)
  - [4.1 lda可视化算子](#41-lda可视化算子)
    - [4.1.1 lda_topic_clustering](#411-lda_topic_clustering)
- [5. 生成算子](#5-生成算子)
  - [5.1 多模态生成算子](#51-多模态生成算子)
    - [5.1.1 generate_qna_for_images](#511-generate_qna_for_images)



--- 
- #1055
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JunnYu JunnYu JunnYu approved these changes

Assignees

No one assigned

Labels

contributor

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@sanbuphy @JunnYu @bigmover