Use Github Actions?

[bayazidbh]

I've recently gotten into uBlue, and it is pretty neat that they have a setup for building their image using Github Actions that's easy to fork and do yourself.

I think this repo could benefit from having a similar process so that we could just fork the repo, change the create-arch-bootstrap.sh (and whichever else we want, if we're willing to fix issues and update changes to utility / offset ourselves), and get our own weekly or daily build of custom conty.sh.

What do you think?

[Kron4ek]

I think it's good idea. I'm currently working on this, will add soon.

[bayazidbh]

Thanks, forked the repo and added my own packages in my create-arch-bootstrap.sh, actions is currently still at Utils CI so I'll check them again later after work.

[Kron4ek]

Note that although it runs weekly (at each Friday), it doesn't start automatically right after you change files in the repo. You can run it manually though at any time by using the "Run workflow" button.

[bayazidbh]

Yeah, I noticed. Friday makes sense -- that should give time for people to check for any issues during the weekend, and make some adjustment if maybe there's a different offset they need to enter (and people can just adjust the cron job if they want a different interval). The utils take a surprisingly long time to run, for a small file, but I guess it just takes time.

[bayazidbh]

Though I noticed that the offset needed to be adjusted from time to time, so maybe at the start of the action, we can do a sed -i "s/utils_size=.*/utils_size=$(stat -c %s ./utils.tar.gz)/" create-conty.sh (or just set utils_size to utils_size=$(stat -c %s ./utils.tar.gz) instead of an explicit value, if it's possible) so that the value there is always up-to-date?

I'm not familiar enough with the project to be able to tell if that might create an issue, though.

[Kron4ek]

Good idea, i think i will add size autocorrection via sed into create-conty.sh.

[bayazidbh]

Schedule Action failed today, likely because of: https://forum.garudalinux.org/t/chaotic-aur-current-status/29161

Annoying, but just need to wait I guess.

[Kron4ek]

Yes, definitiely because of this. Will wait.

Edit. They fixed the problem.

[bayazidbh]

Is it possible to add a workflow to push the Actions artifact to a "Continuous Integrations pre-release" Release as well?

It would help when I'm setting up a new distro and haven't logged into everything, as you can't download Actions artifact without logging in on GIthub. Would also provide a permanent link that I could use with a script to automatically download and extract the new binary. The email it generate would also help in notifying when I should download a new binary as sometimes it took me a day or two to remember.

[Kron4ek]

It works. The problem is that GitHub does not allow to upload assets with sizes greater than 2 GB, i can't increase this limit. However, this can be worked around by splitting the file into multiple pieces and uploading each of them as a separate asset, i can try to implement this if you want.

[Kron4ek]

Implemented in 90f0bfb.

In the continuous release in your repo you will get such files attached: conty.sh_part01, conty.sh_part02. You will need to download them and then combine them into a single file by using cat:

$ cat conty.sh_part01 conty.sh_part02 > conty.sh
$ chmod +x conty.sh

If the file is larger than 4 GB, it will be splitted into even more parts.

[bayazidbh]

Sure, I'm fine with that. The reason I really wanted an auto-release CI is so that I could get a static URL (that also don't require logging in) that I could put inside a script to automatically download, extract, and re-export all the shortcuts.

The CI artifact has been great, but the lack of static URL was a bit annoying, and I wanted an easy way to share it with my friends too.

[Kron4ek]

I understand. Let me know if the new workaround for large assets works for you and that the resulting conty.sh works without issues.

[bayazidbh]

It works great! Sorry, it took a while, I think I either went pass the GH size limit during the build process or an error installing a package (junction, not on chaotic-aur anymore) preventing the image to be uploaded, but I tested it just now - it downloaded, combined, and ran just fine.

Thanks a lot.

[bayazidbh]

Is there anything that I can do to work around the no space left issue, aside for trying to just remove packages one by one until it fits?

https://github.com/bayazidbh/Conty/actions/runs/6361770042/job/17276412568

[bayazidbh]

Thanks, will test now

[bayazidbh]

Hm, I'm not sure if it's an issue with the CI's container space. According to maximize-build-space, I have 55G available, and from my offline build test, I use around 33.5GiB during build process, and including the finished image and Conty.sh, that puts it at 38.5GiB.

The finished Conty image is approximately 3-4.5GiB depending on what I'm including -- let's assume it's 5GiB and that I need another 5GiB for the splitting process (whcih shouldn't be relevent to the non-release CI), that should still leave with free spaces. Even assuming 55G means 55GB, I should only be using around 46.7GB.

So it seems like it's just the accumulation of my allotted GitHub artifact space running out? Or at least being used too much for now. For now I'll just build it offline and let the GH Actions be to see if it'll work again in a week or two.

[Kron4ek]

What's interesting is that it builds Conty with no issues, but then fails on the "Upload artifact" step. You can try to add this line to the workflow file:

sudo rm -rf root.x86_64

So it would look like this:

chmod +x create-arch-bootstrap.sh create-conty.sh
sed -i 's/sed/#sed/g' create-arch-bootstrap.sh
sudo ./create-arch-bootstrap.sh && ./create-conty.sh
sudo rm -rf root.x86_64

[bayazidbh]

I think it might be an issue with upload space for artifact more than anything -- deleting root.x86_64 and image file didn't change anything.

For now I'll just leave it alone, maybe by Friday it'll fix itself and can upload correctly. For now, I already have it manually built on my devices and I can just ignore it for a while.

[bayazidbh]

And suddenly it just works, even though earlier in the week, the other manual runs kept failing, even when the same push.

So it's probably just GitHub free being GitHub free.