Skip to content

tools/ci: Added CI system on Windows Native - #15989

Merged
lupyuen merged 1 commit into
apache:masterfrom
simbit18:simbit18-ci-native
Mar 19, 2025
Merged

tools/ci: Added CI system on Windows Native#15989
lupyuen merged 1 commit into
apache:masterfrom
simbit18:simbit18-ci-native

Conversation

@simbit18

@simbit18 simbit18 commented Mar 13, 2025

Copy link
Copy Markdown
Contributor

Summary

This PR adds support for the CI system for Windows native as well. It allows you to build NuttX on GitHub and test it locally for Windows users.

With these CI tools with PowerShell scripts, it is possible to build NuttX for Windows native using (for now only) Cmake + Ninja with the same logic as the CI system with Bash scripts.

This allows the msvc job to be used not only with the simulator (currently only with Visual Studio 17 2022), but also with other architectures using the same Windows runner to get more coverage and avoid future breakage. As with the other jobs, we use artifacts to save the compilation result at the end of the workflow execution (previously for the simulator it was not done).

The proposed solution is based on the following additions and modified:

Modified Files
buildyml -> only CI Job MSVC

New Files in tools/
ci/cibuild.ps1 -> Run the CI Builds
ci/platforms/windows.ps1 -> Added script for installing toolchains and tools.
Added the pre-built kconfig-frontends that contains my kconfig-tweak script converted to PowerShell (without using the sed and grep tools)

testlist/windows.dat -> Target (Add sim (msvc), risc-v arm)
tools/testbuild.ps1

Impact

Impact on user: This PR adds CI tools for Windows users.

Impact on build: This PR adds CI support NuttX on Windows native to avoid future breakage.

Impact on hardware: NO

Impact on documentation: NO

Impact on security: NO

Impact on compatibility: NO

Testing

We tested the NuttX build on GitHub and locally.

How we build on GitHub and test locally.
GitHub
https://github.com/simbit18/nuttx_test_pr/actions/runs/13839199580/job/38722147529#logs

Locally
cd .\nuttx\tools\ci\

.\cibuild.ps1 -n -i -A -C -N .\testlist\windows.dat

Simulator

C:\nuttx_test\sim\windows\nuttx.exe

NuttShell (NSH)
nsh> uname -a
NuttX  0.0.0 eb1aefdc24-dirty Mar 13 2025 15:28:38 sim sim
nsh> help
help usage:  help [-v] [<cmd>]

    .           cmp         fdinfo      mkdir       pwd         umount
    [           dirname     free        mkfifo      rm          unset
    ?           dd          help        mkrd        rmdir       uptime
    alias       df          hexdump     mount       set         usleep
    unalias     dmesg       ifconfig    mv          sleep       watch
    arp         echo        ifdown      nslookup    source      xd
    basename    env         ifup        pidof       test        wait
    break       exec        kill        poweroff    time
    cat         exit        pkill       quit        true
    cd          expr        losetup     printf      truncate
    cp          false       ls          ps          uname

Builtin Apps:
    nsh       sh        ping      ostest    hello
nsh> poweroff

Arm nucleo-l152re:nsh
arm

@github-actions github-actions Bot added Area: Tooling Area: CI Size: XL The size of the change in this PR is very large. Consider breaking down the PR into smaller pieces. labels Mar 13, 2025
@nuttxpr

nuttxpr commented Mar 13, 2025

Copy link
Copy Markdown

[Experimental Bot, please feedback here]

This PR description mostly meets the NuttX requirements, but could be improved in a few areas for clarity and completeness.

Strengths:

  • Clear Summary of the "Why": The description clearly explains the motivation for the change: enabling native Windows CI.
  • Good Explanation of "How": It outlines the technical changes, including new files and modifications to existing ones.
  • Provides Testing Logs (sort of): Links to a GitHub Actions run and local testing instructions are provided. However, actual log output is minimal.

Areas for Improvement:

  • More Precise Summary of "What": While the overall goal is clear, the summary could be more precise about what functionality is being added. For instance, mention specific build targets (e.g., "Adds CI support for building NuttX natively on Windows using CMake + Ninja, targeting sim, RISC-V, and ARM architectures.").
  • Expand on Impact on Build: Be more specific about build process changes. Does this introduce new build dependencies for Windows users? Does it change anything about how NuttX is configured or built on other platforms?
  • Show Relevant Testing Logs: The provided logs are insufficient. Instead of linking to the entire GitHub Actions run, include snippets of the relevant build and test output for each supported target (sim, RISC-V, ARM). The screenshot for ARM is helpful but should be supplemented by textual log output. The simulator log output is good, but include similar output for the other architectures. Demonstrate that key functionalities work on each target after the change.
  • Specify Build Host Details: The "Testing" section needs more detail about the local build host. Provide specifics about the Windows version, PowerShell version, CMake version, Ninja version, MSVC version, and any other relevant tool versions used.
  • Clarify kconfig-tweak Changes: Mentioning "kconfig-tweak converted to PowerShell" is good, but briefly explain why this conversion was necessary and what specific changes were made (e.g., "Rewrote kconfig-tweak using PowerShell to avoid dependencies on sed and grep, which are not readily available in standard Windows environments.").
  • Consider Documentation Impact: Even if no formal documentation updates are required, consider whether a brief note in the README or contributing guidelines might be helpful for Windows users who want to leverage the new CI capabilities.

Example Improvements to the Testing Section:

## Testing

I confirm that changes are verified on local setup and works as intended:

* Build Host: Windows 11, PowerShell 7.3.0, CMake 3.25, Ninja 1.11.1, MSVC v143 (Visual Studio 2022)
* Targets:
    * sim (msvc): Windows 11
    * risc-v: qemu-rv32 (specify the specific RISC-V target)
    * arm: nucleo-l152re (specify the specific ARM target)

Testing logs **before** change (for each target - demonstrate the previous behavior/error):

<Relevant log snippets showing previous behavior/error for each target>


Testing logs **after** change (for each target - show successful build and basic functionality):

**sim (msvc):**

<Output of 'nuttx.exe' demonstrating basic NSH functionality, similar to what's already provided>


**risc-v:**
```

arm (nucleo-l152re):

<Build output showing successful compilation>
<Serial console output showing boot and basic functionality.  Include more than just the screenshot.>


By addressing these points, the PR description will be more comprehensive and easier for reviewers to assess the changes thoroughly. Remember, clarity and completeness are crucial for a smooth review process and successful integration of your contributions.

@hartmannathan hartmannathan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't test this, since I don't have a Windows setup, but I tried to review as well as possible by reading the code.

The only real suggestion I can make is to include a one-line hint in each file's header. The coding standard (see https://nuttx.apache.org/docs/latest/contributing/coding_style.html#file-organization) allows for "An optional, one-line description of the file contents." I would suggest: "PowerShell script for CI on Windows Native" or something to that effect. Because Unix-heads (like me!) will likely wonder what these files and pswd are, when seen outside the context of this PR.

(Actually, I would recommend the same suggestion for all files throughout NuttX. It can be hard to understand what different files do and a tiny hint like this can go a long way.)

Thanks for improving CI and Windows support!

#!/usr/bin/env pswd
############################################################################
#
#

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this empty comment line meant to be doubled?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added file name and description

Comment thread tools/ci/testlist/windows.dat Outdated
This PR adds support for the CI system for native Windows as well. It allows you to build NuttX on GitHub and test it locally for Windows users.

With these CI tools with PowerShell scripts, it is possible to build NuttX for Windows Native using (for now only) Cmake + Ninja with the same logic as the CI system with Bash scripts.

This allows the msvc job to be used not only with the simulator (currently only with Visual Studio 17 2022), but also with other architectures using the same Windows runner to get more coverage and avoid future breakage.
As with the other jobs, we use artifacts to save the compilation result at the end of the workflow execution (previously for the simulator it was not done).

The proposed solution is based on the following additions and modified:

Modified Files
buildyml -> only CI Jobs MSVC

New Files in tools/
ci/cibuild.ps1 -> Added Powershell script for Run the CI Builds
ci/platforms/windows.ps1 -> Added Powershell script for installing toolchains and tools.
testlist/windows.dat -> Target (Add sim (msvc), risc-v arm)
tools/testbuild.ps1

We tested the NuttX build on GitHub and locally.

How we build on GitHub and test locally.

Locally
cd .\nuttx\tools\ci\

.\cibuild.ps1 -n -i -A -C -N .\testlist\windows.dat

Signed-off-by: simbit18 <simbit18@gmail.com>
@simbit18
simbit18 force-pushed the simbit18-ci-native branch from eb1aefd to 266da68 Compare March 14, 2025 09:41
@simbit18

Copy link
Copy Markdown
Contributor Author

Hi @hartmannathan done

@simbit18

Copy link
Copy Markdown
Contributor Author

Hi @lupyuen perhaps before a future merge of this PR for consistency it is necessary to rename the job from msvc to Windows what do you think ?

@lupyuen

lupyuen commented Mar 14, 2025

Copy link
Copy Markdown
Member

@simbit18 If we rename msvc toWindows, then msys2 will sound confusing, since it's also Windows? 🤔

@simbit18

simbit18 commented Mar 14, 2025

Copy link
Copy Markdown
Contributor Author

@lupyuen maybe WinNative ?

MSVC is -> Microsoft Visual C++
https://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B

@lupyuen

lupyuen commented Mar 14, 2025

Copy link
Copy Markdown
Member

@simbit18 Maybe we can rename it later? It might confuse the folks who are watching the builds. Actually msvc vs msys2 kinda makes sense, for nostalgic reasons :-)

@hartmannathan hartmannathan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I can't test (no Windows setup) but looks good on code reading. Thanks for improving NuttX CI!

Comment thread tools/ci/platforms/windows.ps1
Comment thread tools/ci/platforms/windows.ps1
@simbit18

simbit18 commented Mar 19, 2025

Copy link
Copy Markdown
Contributor Author

Sorry @xiaoxiang781216 @anchao @xuxin930 since you have added simulator building with Visual Studio 17 2022 with CMake, could you please see this PR and have your comments?

@lupyuen
lupyuen merged commit 34c4b15 into apache:master Mar 19, 2025
@xuxin930

Copy link
Copy Markdown
Contributor

@simbit18 excellent work!!!!

simbit18 added a commit to simbit18/nuttx-apps that referenced this pull request Mar 19, 2025
added
    CI: Kill CI Test after 2 hours
    see apache/nuttx#14849

    job msvc Windows native
    see
    apache/nuttx#13894
    apache/nuttx#15989

Signed-off-by: simbit18 <simbit18@gmail.com>
@simbit18
simbit18 deleted the simbit18-ci-native branch March 20, 2025 14:20
simbit18 added a commit to simbit18/nuttx-apps that referenced this pull request Mar 20, 2025
added
    CI: Kill CI Test after 2 hours
    see apache/nuttx#14849

    job msvc Windows native
    see
    apache/nuttx#13894
    apache/nuttx#15989

Signed-off-by: simbit18 <simbit18@gmail.com>
simbit18 added a commit to simbit18/nuttx-apps that referenced this pull request Mar 20, 2025
added
    CI: Kill CI Test after 2 hours
    see apache/nuttx#14849

    job msvc Windows native
    see
    apache/nuttx#13894
    apache/nuttx#15989

Signed-off-by: simbit18 <simbit18@gmail.com>
lupyuen pushed a commit to apache/nuttx-apps that referenced this pull request Mar 20, 2025
added
    CI: Kill CI Test after 2 hours
    see apache/nuttx#14849

    job msvc Windows native
    see
    apache/nuttx#13894
    apache/nuttx#15989

Signed-off-by: simbit18 <simbit18@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: CI Area: Tooling Size: XL The size of the change in this PR is very large. Consider breaking down the PR into smaller pieces.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants