added test with wizardLM-7B and oobabooga webui

[drumnicorn]

my pc cant run good enough models but here is the code.

[Sentdex]

How is

MODEL_PATH = "models/wizardLM-7B-HF" meant to work?

This would mean something like:

wizard_model = GPT2LMHeadModel.from_pretrained("models/wizardLM-7B-HF")

This fails for me since there's nothing there. I would normally use transformers more like

wizard_model = GPT2LMHeadModel.from_pretrained("TheBloke/wizardLM-7B-HF"), which either pulls from cached models or downloads it. Never seen it in the way you've proposed, any reason for this?

edit: formatting

[drumnicorn]

I have a folder with all my local models linked to the "models" dir in my pc. I downloaded the wizard 7b hf model from huggingface

https://huggingface.co/TheBloke/wizardLM-7B-HF

[userbox020]

Hello, i over reacted, reading now answer. I would like to help in anything i can

[drumnicorn]

Hello, i over reacted, reading now answer. I would like to help in anything i can

                   -`                    roggen@archlinux 
                  .o+`                   ---------------- 
                 `ooo/                   OS: Arch Linux x86_64 
                `+oooo:                  Kernel: 6.3.5-arch1-1 
               `+oooooo:                 Uptime: 12 mins 
               -+oooooo+:                Packages: 1112 (pacman) 
             `/:-:++oooo+:               Shell: bash 5.1.16 
            `/++++/+++++++:              Resolution: 1920x1080, 1080x1920 
           `/++++++++++++++:             DE: Plasma 5.27.5 
          `/+++ooooooooooooo/`           WM: KWin 
         ./ooosssso++osssssso+`          Theme: [Plasma], Breeze [GTK2/3] 
        .oossssso-````/ossssss+`         Icons: Papirus [Plasma], Papirus [GTK2/3] 
       -osssssso.      :ssssssso.        Terminal: konsole 
      :osssssss/        osssso+++.       CPU: Intel i7-7700K (8) @ 4.600GHz 
     /ossssssss/        +ssssooo/-       GPU: NVIDIA GeForce GTX 1060 6GB 
   `/ossssso+/:-        -:/+osssso+-     Memory: 7253MiB / 15939MiB 
  `+sso+:-`                 `.-/+oso:
 `++:.                           `-/+/                           
 .`                                 `/                           

I think should work with 8GB vram

edit: currently my memory overloads when i try to init the model. I am not sure about this because my Memory seems to overload not my vram. But in any case i still get stuck at loading the model with my setup.

PS: if anyone knows a way to run this models on my setup i would be very happy :D

[userbox020]

i have amd videocard bro, so im stuck with the torch configuraiton to run LLM models
Also i didnt understand Sentdex how to make TermGPT do what i want. It just keeping giving back motivational and goodbye responses the gpt3-turbo

[drumnicorn]

I created an issue for you, this is a pull request better watch video or research how contributions on GitHub work @userbox020

[userbox020]

Thanks bro i apreciatte your patient i'm not even English speaker, thats means Lot for me