Skip to content

Fix building package without a GPU #2049

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mrwyattii merged 3 commits into from
Jul 6, 2022
Merged

Fix building package without a GPU #2049

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4b20a92
Check for bf16 support only if CUDA is available
aphedges Jun 24, 2022
ff284d2
Merge branch 'master' into 2010-fix-no-gpu-build
mrwyattii Jul 6, 2022
2227899
Merge branch 'master' into 2010-fix-no-gpu-build
mrwyattii Jul 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -235,7 +235,7 @@ def create_dir_symlink(src, dest):
nccl_version = ".".join(str(torch.cuda.nccl.version())[:2])
else:
nccl_version = ".".join(map(str, torch.cuda.nccl.version()[:2]))
if hasattr(torch.cuda, 'is_bf16_supported'):
if hasattr(torch.cuda, 'is_bf16_supported') and torch.cuda.is_available():
bf16_support = torch.cuda.is_bf16_supported()

@mrwyattii mrwyattii Jun 28, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bf16_support value is only used in unit testing - perhaps we could add a try...except block around this?

@aphedges aphedges Jun 28, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My first commit would be sufficient to prevent the crash because it checks whether CUDA is available.

However, that doesn't solve the problem of building and running on different hardware with different bf16 support.

@mrwyattii mrwyattii Jun 29, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bf16 support check is only utilized in unit tests - so it shouldn't affect users that build/run on different hardware

@aphedges aphedges Jun 29, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know it's set up that way in CI, but if a developer runs the unit tests on a different machine than the build, it could still cause problems.

if torch_available and hasattr(torch.version, 'hip') and torch.version.hip is not None:
hip_version = ".".join(torch.version.hip.split('.')[:2])
Expand Down