Skip to content

Set build user's uid when creating Migraphx/ROCM docker images - #23657

Merged
Changming Sun (snnn) merged 9 commits into
mainfrom
snnn/ci
Feb 21, 2025
Merged

Set build user's uid when creating Migraphx/ROCM docker images#23657
Changming Sun (snnn) merged 9 commits into
mainfrom
snnn/ci

Conversation

@snnn

@snnn Changming Sun (snnn) commented Feb 12, 2025

Copy link
Copy Markdown
Contributor

Description

Set build user's uid when creating Migraphx/ROCM docker images

Motivation and Context

The two pipelines have a serious issue: the docker image used for building/testing the code could come from any branch! Because the machine has 8 GPUs. We don't want to waste the GPUs, so we run multiple ADO agents there to run build pipelines in parallel. However, there is only one docker daemon.
Therefore, last time when I made a change to the ROCM pipeline's docker image I made an error there but I still saw the pipeline passed. Then, later on after I checked in the change the pipeline started failing. It is a chaos.
This PR couldn't address the fundamental issue. It just fixes the mistake I introduced.

@snnn
Changming Sun (snnn) marked this pull request as draft February 12, 2025 01:10
@snnn

Copy link
Copy Markdown
Contributor Author

/azp run ONNX Runtime Web CI Pipeline

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@snnn
Changming Sun (snnn) force-pushed the snnn/ci branch 2 times, most recently from ac77f27 to 9333122 Compare February 12, 2025 04:02
@snnn Changming Sun (snnn) changed the title Set build user's uid when creating Migraphx/ROCM docker images Revert the change that added vcpkg to ROCM CI pipeline Feb 14, 2025
@snnn

Copy link
Copy Markdown
Contributor Author

I bet: if I find a time when nobody uses this pipeline to submit this PR, then everything will pass.

@snnn Changming Sun (snnn) changed the title Revert the change that added vcpkg to ROCM CI pipeline Set build user's uid when creating Migraphx/ROCM docker images Feb 18, 2025
@snnn
Changming Sun (snnn) marked this pull request as ready for review February 18, 2025 04:21
jingyanwangms
jingyanwangms previously approved these changes Feb 18, 2025

@jingyanwangms jingyanwangms left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@snnn

Copy link
Copy Markdown
Contributor Author

/azp run Linux GPU CI Pipeline, Linux MIGraphX CI Pipeline

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 2 pipeline(s).

@tianleiwu

Copy link
Copy Markdown
Contributor

The pipeline uses current user ID to run docker (and the user id might not exists in docker image):

How does BUILD_UID=1004 help?

@snnn

Changming Sun (snnn) commented Feb 18, 2025

Copy link
Copy Markdown
Contributor Author

How does BUILD_UID=1004 help?

I bet the UID is always 1004. I added two commands at:

python --version; id ; ls -lha /home ; \

In the log it shows the uid is 1004. And the /home/onnxruntimedev folder is owned by the onnxruntimedev user, so, in theory it should work. But it is not working.

Here is a log:
https://dev.azure.com/onnxruntime/onnxruntime/_build/results?buildId=1620518&view=logs&j=f2f63060-d9d6-52d0-adee-b97db5a9ab91&t=7d92004f-43ca-5c62-0a5c-970cbb2d24e8

When we built the image, we have:

adduser --gecos 'onnxruntime Build User' --disabled-password onnxruntimedev --uid 1004

@tianleiwu

Tianlei Wu (tianleiwu) commented Feb 18, 2025

Copy link
Copy Markdown
Contributor

In the log it shows the uid is 1004. And the /home/onnxruntimedev folder is owned by the onnxruntimedev user, so, in theory it should work. But it is not working.

Shall we also add group id using addgroup?

@amarin16

Copy link
Copy Markdown
Contributor

/azp run Linux GPU CI Pipeline, Linux MIGraphX CI Pipeline

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 2 pipeline(s).

Changming Sun added 2 commits February 19, 2025 17:06
amarin16
amarin16 previously approved these changes Feb 20, 2025
Changming Sun added 2 commits February 20, 2025 16:05
Changming Sun added 2 commits February 21, 2025 07:46
@snnn
Changming Sun (snnn) merged commit 98511b0 into main Feb 21, 2025
@snnn
Changming Sun (snnn) deleted the snnn/ci branch February 21, 2025 22:08
Guenther Schmuelling (guschmue) pushed a commit that referenced this pull request Mar 6, 2025
### Description
Set build user's uid when creating Migraphx/ROCM docker images
Ashrit Shetty (ashrit-ms) pushed a commit that referenced this pull request Mar 17, 2025
### Description
Set build user's uid when creating Migraphx/ROCM docker images
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants