Stuart Feedback

[makubacki]

A tool called Stuart is integrated with edk2 to help provide a consistent build and dependency management interface across different user environments.

Quick references:

• Repo that hosts Stuart - edk2-pytool-extensions
• User documentation area - User Documentation
• TianoCore Specific Stuart Build Instructions - How to Build with Stuart

Feedback has been received on numerous occasions that Stuart:

1. Does not integrate well into some developer workflows - particularly on Linux
2. Has dependencies (like .NET) that are complicated to install - especially on various Linux distros
3. Has a CLI that is difficult to understand/use
4. Has a configuration process that is difficult to understand/use
5. Has a poor implementation
6. Is redundant/unnecessary

and so on...

This discussion was created to gather constructive feedback that can be channeled into actionable improvements. The tool aspires to provide a Python-based build environment that can accommodate common developer flows in the edk2 project, and this feedback will hopefully help bring it closer to achieving that.

---

I understand it can be frustrating to use something that may have fallen short of expectations. However, please try to keep feedback constructive, concise, and respectful.

[bcran]

One issue for me is having to pass the toolchain using TOOL_CHAIN_TAG= instead of -t (related: -t appears to have been redefined to mean "target" instead of "toolchain").
Also, specifying the build type using TARGET={RELEASE,NOOPT,DEBUG} doesn't appear to be documented.

Errors aren't always propagated:

(venv) bcran@cube:~/src/uefi/edk2$ stuart_update -c .pytool/CISettings.py
...
SECTION - 	Updated/Verified 3 dependencies
SECTION - Second pass update of environment
ERROR - Could not find appropriate folder for mu-uncrustify-release. Host(os='Linux', arch='x86', bit='64')
ERROR - Could not find appropriate folder for mu_nasm. Host(os='Linux', arch='x86', bit='64')
ERROR - Could not find appropriate folder for edk2-acpica-iasl. Host(os='Linux', arch='x86', bit='64')
ERROR - Could not find appropriate folder for mu-uncrustify-release. Host(os='Linux', arch='x86', bit='64')
ERROR - Could not find appropriate folder for mu_nasm. Host(os='Linux', arch='x86', bit='64')
ERROR - Could not find appropriate folder for edk2-acpica-iasl. Host(os='Linux', arch='x86', bit='64')
ERROR - Could not find appropriate folder for mu-uncrustify-release. Host(os='Linux', arch='x86', bit='64')
ERROR - Could not find appropriate folder for mu_nasm. Host(os='Linux', arch='x86', bit='64')
ERROR - Could not find appropriate folder for edk2-acpica-iasl. Host(os='Linux', arch='x86', bit='64')
Updating. Done
SECTION - 	Updated/Verified 3 dependencies
SECTION - Summary
PROGRESS - Success

Also, when using Edk2ToolsBuild.py, if I remove the brotli submodule code:

(venv) bcran@cube:~/src/uefi/tmp/edk2$ python3 BaseTools/Edk2ToolsBuild.py -t GCC5
SECTION - Init Self Describing Environment
SECTION - Loading Plugins
SECTION - Start Invocable Tool
Traceback (most recent call last):
  File "/home/bcran/src/uefi/tmp/edk2/BaseTools/Edk2ToolsBuild.py", line 177, in <module>
    main()
  File "/home/bcran/src/uefi/tmp/edk2/BaseTools/Edk2ToolsBuild.py", line 173, in main
    Edk2ToolsBuild().Invoke()
  File "/home/bcran/src/uefi/tmp/edk2/venv/lib/python3.11/site-packages/edk2toolext/base_abstract_invocable.py", line 236, in Invoke
    retcode = self.Go()
              ^^^^^^^^^
  File "/home/bcran/src/uefi/tmp/edk2/BaseTools/Edk2ToolsBuild.py", line 148, in Go
    raise Exception("Failed to build.")
Exception: Failed to build.

Looking in BaseTools/BaseToolsBuild/BASETOOLS_BUILD.txt there are no ERROR lines, even at:

INFO - BrotliCompress.c:20:10: fatal error: ./brotli/c/common/constants.h: No such file or directory
INFO -    20 | #include "./brotli/c/common/constants.h"
INFO -       |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
INFO - compilation terminated.

[makubacki]

Thanks for checking. An issue here is that the GNUmakefile errors need to be recognized and logged as ERROR. I filed a bug to track it - tianocore/edk2-pytool-extensions#539.

[makubacki]

One issue for me is having to pass the toolchain using TOOL_CHAIN_TAG= instead of -t (related: -t appears to have been redefined to mean "target" instead of "toolchain"). Also, specifying the build type using TARGET={RELEASE,NOOPT,DEBUG} doesn't appear to be documented.

You can eventually see some help:

(venv3.10) ➜  edk2 git:(master) ✗ stuart_ci_build -c .pytool/CISettings.py --help
usage: stuart_ci_build [-h] [-p PACKAGELIST] [-a REQUESTED_ARCH] [-t REQUESTED_TARGET] [-force_piptools | -no_piptools]
                       [--build-config BUILD_CONFIG] [--verbose]

options:
  -h, --help            show this help message and exit
  -p PACKAGELIST, --pkg PACKAGELIST, --pkg-dir PACKAGELIST
                        Optional - A package or folder you want to update (workspace relative).Can list multiple by doing -p
                        <pkg1>,<pkg2> or -p <pkg3> -p <pkg4>
  -a REQUESTED_ARCH, --arch REQUESTED_ARCH
                        Optional - CSV of architecutres requested to update. Example: -a X64,AARCH64
  -t REQUESTED_TARGET, --target REQUESTED_TARGET
                        Optional - CSV of targets requested to update. Example: -t DEBUG,NOOPT
  -force_piptools, --fpt
                        Force the system to use pip tools
  -no_piptools, --npt   Force the system to not use pip tools
  --build-config BUILD_CONFIG
                        Provide shell variables in a file
  --verbose, --VERBOSE, -v
                        verbose

CLI Env Guide:
  <key>=<value>              - Set an env variable for the pre/post build process
  <key>                      - Set a non-valued env variable for the pre/post build process
  BLD_*_<key>=<value>        - Set a build flag for all build types
                               (key=value will get passed to build process)
  BLD_*_<key>                - Set a non-valued build flag for all build types
  BLD_<TARGET>_<key>=<value> - Set a build flag for build type of <target>
                               (key=value will get passed to build process for given build type)
  BLD_<TARGET>_<key>         - Set a non-valued build flag for a build type of <target>

That also works for other commands supported by .pytool/CISettings.py like stuart_ci_setup.

Note that I had to know to use stuart_ci_build instead of stuart_build (because this is a "CI" file - .pytool/CISettings.py). If I had tried to use stuart_build:

(venv3.10) ➜  edk2 git:(master) ✗ stuart_build -c .pytool/CISettings.py --help 
Unable to use .pytool/CISettings.py as a BuildSettingsManager
Did you mean to use a different kind of invocable?
Traceback (most recent call last):
  File "/home/mikuback/src/edk2/venv3.10/lib/python3.10/site-packages/edk2toolext/edk2_invocable.py", line 360, in ParseCommandLineOptions
    self.PlatformSettings = locate_class_in_module(
TypeError: 'NoneType' object is not callable

...<Python traceback continues on for a while>

The text "Did you mean to use a different kind of invocable?" is a hint but probably not useful to someone looking for help.

Did finding this information contribute to the issue you were having? Is the information helpful?

[makubacki]

Issue filed for -t confusion: tianocore/edk2-pytool-extensions#540

[Javagedes]

tianocore/edk2-pytool-extensions#540 has been resolved. Additional error logging has been added when either TOOL_CHAIN_TAG or TARGET has not been set, along with a graceful exit. This will be available when v0.23.3 is released.

[bcran]

Thanks!

[bcran]

@makubacki I was wondering if you could take a look at my stuart_pr_eval failure at tianocore/edk2-pytool-extensions#550 ? I don't know why it's not working.

[makubacki]

I replied. Feel free to follow up there if more help is needed.

[osteffenrh]

• Does not integrate well into some developer workflows - particularly on Linux

A wish of mine would be an "offline" mode, where it does not download the toolchain (gcc, nasm, etc) but uses the tools it can find in PATH. That is useful for Linux distribution specific work where we want to use the tools of the distro.

I started looking into this a while back but then got distracted.

• Has dependencies (like .NET) that are complicated to install - especially on various Linux distros

I get that .NET is convenient in come cases and it is easily available on most Linux environments, but it is a large dependency compared to what it is used for in the end (downloading dependencies via nuget, I think? Or does it do other things too?). It adds quite some size to the container images that we use in the (Linux) CI.

[makubacki]

Offline mode seems reasonable, I've heard it before. We're also reducing .NET usage. @Javagedes will follow up.

[spbrogan]

@osteffenrh - the online/offline and even .NET dependency comes down to how a platform uses stuart. For example, if you don't leverage any nuget package based dependencies then you don't actually need .NET or nuget. As @makubacki mentioned because this has been a pain point for some edk2 developers we are trying to move edk2 away from nuget based dependencies.

The platform controls the dependencies and tools it uses. You can control this through the scopes. You could easily create an option for an edk2 platform (like ovmf, armvirt, emulatorpkg) or your own platform that disables the scopes and makes it work with tools on the path. I don't believe this is a scalable or sustainable practice in that an end user has no idea what versions of tools are supported and thus we didn't do that when setting up the CI platforms in Edk2. One of the best value points for pytools is that it lets you define a repeatable and reproducible environment using artifacts that are checked into the source tree. But regardless, pytools shouldn't need any changes to support offline mode. Again, I am not a fan of this but can understand it could be ideal or useful for some projects and pytools in general should not require online usage if a "platform" doesn't require it. Finally, since I am not sure we have built a platform using all external tools (in a long time) it is possible something has creeped in. If so please let us know so we can fix it.

[Javagedes]

As requested from @mdkinney, We've added some stuart CLI usability improvements and will leave them open for a few days for anyone to leave comments.

Review each PR to see the changes, but overall they do the following:

• When using the -h flag, Print the available options for -t (target), -p (package), and -a (architecture) if a configuration file has been provided.
• Return a clean error if you provide an invalid command line argument (to include -<flag>, --<flag>, or /<flag>)
• Allow running a subset of tests with stuart_ci_build by setting -d or --disable-all and then turning back on the plugin you wish to run via the command line with <Plugin>=run

tianocore/edk2-pytool-extensions#753
tianocore/edk2-pytool-extensions#754
tianocore/edk2-pytool-extensions#755