[chizuchizu]

Added training scripts Independent of Hugging Face's Trainer

Description

Purpose of the PR

• Added training scripts that don't depend on Hugging Face's Trainer class.

Related Issue(s)

• Nothing.

Changes

• Added train_ds.py
• Added utils scripts imported from Microsoft VisualChat.
• Added the training config and the training shell script.
• Update documentation on training and inference methods.

Model/Algorithm Performance

• Trained the GIT model following projects/opt/exp002_ds.yml with both ZeRO-2 and ZeRO-3, confirming valid inference.
• Trained Video BLIP with ZeRO-3.
• Observed an increase in GPU memory consumption compared to when using Trainer.

Dependencies

• Nothing.

Reviewer Notes

Confirmed

• I have updated the documentation accordingly.
• I have adhered to the coding standards and guidelines of this project.
• I have added comments, especially in hard-to-understand areas.