[BOUNTY - $200] Windows native support

[AlexCheema]

• Windows is largely untested. Don't know of anyone who has it working natively, only in WSL
• Known issues on windows: Error on Windows 11: "NotImplementedError" in asyncio #184 , tinygrad not officially supported
• Perhaps llama.cpp backend [BOUNTY - $500] Llama.cpp inference engine #167 will enable working inference on windows if tinygrad doesn't work or PyTorch (working on [Bounty] PyTorch & HuggingFace Interface #139)

[omarei-omoto]

PLEAAASE THIS

[Shivp1413]

I made changes in main.py file, and now its working for windows as well. But I get following error:
Error: Failed to fetch completions: Error processing prompt (see logs with DEBUG>=2): No module named '_posixshmem'
main.zip

[mercurymason]

The '_posixshmem' module is associated with POSIX shared memory, which afaik is not available on Windows. This error due to tinygrad's reliance on Unix-specific features. It doesn't seem like this problem will be soon resolved by tinygrad, as they have discontinued official Windows support in tinygrad version 0.7.0.

WSL would likely enable tinygrad compatibility on Windows, but kind of breaks the native in "native support". As @AlexCheema mentioned, llama.ccp or pytorch backend may provide a fix -- also, JAX is compatible with Windows.

[tensorsofthewall]

@AlexCheema can you assign this to me? I can try to implement this

[luckyai490]

@AlexCheema I will solve this issue. Please assign this issue to me.

[codebuster22]

I'm willing to add to the bounty amount. 1,000 USD (in USDC/USDT).

Need this done. Even getting it working with WSL properly is also much appreciated.

[Dead-Bytes]

@AlexCheema we can solve this by HuggingFaceInference Engine as i used transformers and torch for compute which can be easily run on windows

[generalconsensus]

Where are we with this???

[Dead-Bytes]

also from huggingface face inference engine this is solved