How to upload a large .pkl file to GitHub — not working via Git or manually

[rajendradayma]

Discussion Type

Question

Discussion Content

Hi everyone,

I'm trying to upload a large .pkl (pickle) file to my GitHub repository, but facing persistent issues:

I tried uploading manually via the GitHub web interface — but it fails due to size limits.

I attempted pushing via Git — but the push gets rejected or stalls.

I also tried using Git LFS, followed the setup instructions (git lfs install, git lfs track "*.pkl", etc.), committed and pushed — but still the file either doesn’t upload or causes issues.

Is there something I'm missing with Git LFS? Are there alternative approaches for storing large model or data files (like .pkl) on GitHub?

Any guidance or clarification would be appreciated.

Thanks!

[github-actions[bot]]

🕒 Discussion Activity Reminder 🕒

This Discussion has been labeled as dormant by an automated system for having no activity in the last 60 days. Please consider one the following actions:

1️⃣ Close as Out of Date: If the topic is no longer relevant, close the Discussion as out of date at the bottom of the page.

2️⃣ Provide More Information: Share additional details or context — or let the community know if you've found a solution on your own.

3️⃣ Mark a Reply as Answer: If your question has been answered by a reply, mark the most helpful reply as the solution.

Note: This dormant notification will only apply to Discussions with the Question label. To learn more, see our recent announcement.

Thank you for helping bring this Discussion to a resolution! 💬

[devenkumar1]

Upload a Large .pkl File to GitHub Using Git LFS

1. Install Git LFS:

   git lfs install  
   

2. Track .pkl Files:

       git lfs track "*.pkl"  
   

3. Add the .gitattributes File:

   git add .gitattributes 
   

4.Add and Commit the .pkl File:

5. git add yourfile.pkl

   git commit -m "Add large .pkl file"
   
   

6. Push to GitHub:

    git push origin main  

[Unknown-TM]

Hugging Face Hub (best for ML models)

1. Install CLI :

pip install huggingface_hub
huggingface-cli login

2. Create a model repo on huggingface.co and Upload your .pkl :
   huggingface-cli upload ./model.pkl --repo your-username/your-model

3. Add a script in your GitHub repo :

from huggingface_hub import hf_hub_download
model_path = hf_hub_download("your-username/your-model", "model.pkl")

[github-actions[bot]]

🕒 Discussion Activity Reminder 🕒

This Discussion has been labeled as dormant by an automated system for having no activity in the last 60 days. Please consider one the following actions:

1️⃣ Close as Out of Date: If the topic is no longer relevant, close the Discussion as out of date at the bottom of the page.

2️⃣ Provide More Information: Share additional details or context — or let the community know if you've found a solution on your own.

3️⃣ Mark a Reply as Answer: If your question has been answered by a reply, mark the most helpful reply as the solution.

Note: This dormant notification will only apply to Discussions with the Question label. To learn more, see our recent announcement.

Thank you for helping bring this Discussion to a resolution! 💬

[SaurabhThakulla]

Please like my answer i just want the badge :(

[github-actions[bot]]

🕒 Discussion Activity Reminder 🕒

This Discussion has been labeled as dormant by an automated system for having no activity in the last 60 days. Please consider one the following actions:

1️⃣ Close as Out of Date: If the topic is no longer relevant, close the Discussion as out of date at the bottom of the page.

2️⃣ Provide More Information: Share additional details or context — or let the community know if you've found a solution on your own.

3️⃣ Mark a Reply as Answer: If your question has been answered by a reply, mark the most helpful reply as the solution.

Note: This dormant notification will only apply to Discussions with the Question label. To learn more, see our recent announcement.

Thank you for helping bring this Discussion to a resolution! 💬