CI 2 minute window

Alright, I've got a Gitlab CI pipeline I'm reasonably pleased with as a result of yesterday's efforts. There's 1 glaring problem with it though. The pipeline is triggered by pushing or creating a new tag. At this point in time in the codebase the manifest would have been updated with a new version and download url pointing at the release that will be created. The CI job takes ~2 minutes to complete. During that 2 minute window, anyone who tries to update the package gets an error "Not Found" because the release has no assets yet. I'm interested in peoples' workarounds for this issue.
41 Replies
Calego
Calego3y ago
GitLab
.gitlab-ci.yml · addCiWorkflow · A Krigline (Calego) / Foundry VTT ...
An implementation of the Dungeons & Dragons 5th Edition game system for Foundry Virtual Tabletop (http://foundryvtt.com). This work is permitted under the Open...
Calego
Calego3y ago
This is particularly frustrating because on GH, this problem doesn't exist. I would point the package manifest at releases/latest and that wouldn't exist until after the job ran. Some options I can think of: 1. Commit the new download url and version as part of the CI job. 2. Figure out some way to mock a "latest" release with a stable url to host the manifest.
Unknown User
Unknown User3y ago
Message Not Public
Sign In & Join Server To View
Calego
Calego3y ago
a latest tag which is re-pointed at the newest one is interesting
Unknown User
Unknown User3y ago
Message Not Public
Sign In & Join Server To View
Calego
Calego3y ago
releases and tags are so confusing...
Unknown User
Unknown User3y ago
Message Not Public
Sign In & Join Server To View
Calego
Calego3y ago
yeah that sounds like an order of operations would fix
Unknown User
Unknown User3y ago
Message Not Public
Sign In & Join Server To View
Calego
Calego3y ago
right, I'll see if I can mock that up If this were any package but dnd5e I might not care and say it's good enough to have a 2 minute window.
Unknown User
Unknown User3y ago
Message Not Public
Sign In & Join Server To View
Calego
Calego3y ago
Thought I am concerned about the Bazaar, it might hiccup PF2e has this:
"manifest": "https://gitlab.com/hooking/foundry-vtt---pathfinder-2e/-/jobs/artifacts/release-0.8.x/raw/system.json?job=build",
"manifest": "https://gitlab.com/hooking/foundry-vtt---pathfinder-2e/-/jobs/artifacts/release-0.8.x/raw/system.json?job=build",
Unknown User
Unknown User3y ago
Message Not Public
Sign In & Join Server To View
Calego
Calego3y ago
So that looks like it's pointed at the artifacts of a job on a branch? not per tag
Unknown User
Unknown User3y ago
Message Not Public
Sign In & Join Server To View
Calego
Calego3y ago
Ok they run this on push to their main branch so that'd be a branch pipeline, where mine is a tag pipeline
Unknown User
Unknown User3y ago
Message Not Public
Sign In & Join Server To View
Calego
Calego3y ago
hahaha so sorry we're adding compendia <-> json in the current dnd5e milestone. That's tipped the scales towards "automated release CI" and I'm jumping on this opportunity to figure out gitlab ci
Unknown User
Unknown User3y ago
Message Not Public
Sign In & Join Server To View
Calego
Calego3y ago
We'll have both committed, but we only want to distribute the db files in the zip. And we want to ensure they're the most recently compiled at time of release.
Unknown User
Unknown User3y ago
Message Not Public
Sign In & Join Server To View
Calego
Calego3y ago
dramatically easier to diff and therefore maintain
Unknown User
Unknown User3y ago
Message Not Public
Sign In & Join Server To View
Calego
Calego3y ago
plus, fixing typos could be done in-json and wouldn't need to open foundry at all to contribute
Unknown User
Unknown User3y ago
Message Not Public
Sign In & Join Server To View
Calego
Calego3y ago
something like that, I think we ended up passing things through nedb package for extra safety
Unknown User
Unknown User3y ago
Message Not Public
Sign In & Join Server To View
Unknown User
Unknown User3y ago
Message Not Public
Sign In & Join Server To View
Calego
Calego3y ago
if you're interested that's all Arbron's baby
Ethaks
Ethaks3y ago
Pretty-printed JSON also allows for multiple people to edit different parts of a json file without them interfering with each other (e.g. a migration + a typo fix etc.)
Calego
Calego3y ago
oh i maybe just did a bad does pushing a tag in a ci job trigger a ci job? we'll find out! the push failed cause it didn't have permission to push... dafaq gitlab omg i have to use a PAT to push in a CI job are you for real gitlab?
Daniel Thorp
Daniel Thorp3y ago
Maybe could drop some info on the dnd5e team's use-case for this? This would really help content creators working in groups who don't have the technical knowledge to set up the CI themselves World Smiths is also very interested in getting this set up The issue with Atomizer is that the IDs can change which makes it a suboptimal solution. I would be curious if you found a solution to this?
Calego
Calego3y ago
I'm honestly not sure what the details of the merged solution are, you could holler at arbron in #dnd5e-dev probs
Daniel Thorp
Daniel Thorp3y ago
Can I ping them here? So I don't have to repeat myself xD
Calego
Calego3y ago
Not exactly related to the ci topic tho
Unknown User
Unknown User3y ago
Message Not Public
Sign In & Join Server To View
Calego
Calego3y ago
Ahhhhh interesting! Aaaand project access tokens aren't available for free gitlab.com users. 😬 this is really frustrating coming from github where all of this 'just works'... I also can do something like this:
workflow:
# Run this workflow when a "release-" tag is pushed or created
rules:
- if: $CI_COMMIT_TAG =~ /^release-/
workflow:
# Run this workflow when a "release-" tag is pushed or created
rules:
- if: $CI_COMMIT_TAG =~ /^release-/