mirror of
https://gitea.com/gitea/act_runner.git
synced 2026-07-27 09:57:30 +00:00
ci: mirror release artifacts to Cloudflare R2 (#1114)
Mirrors every release artifact into Cloudflare R2 alongside the existing AWS S3 upload, so both buckets carry the same objects during a parallel period. S3 is untouched and stays authoritative for now; once R2 is confirmed complete it can be dropped by deleting the `blobs:` block and pointing this at R2 alone. ### Why `publishers:` and not a second `blobs:` entry goreleaser's blob pipe authenticates from the global `AWS_*` environment and has no per-entry credential fields, so two different credential sets (AWS S3 and Cloudflare R2) cannot coexist in `blobs:`. Custom publishers do support per-entry `env:`, which is the supported way to isolate the two. ### Why `curl --aws-sigv4` and not `aws`/`rclone` The CI image `docker.gitea.com/runner-images:ubuntu-latest` ships none of `aws`, `rclone`, `s3cmd` or `mc`, but does ship curl 8.5 with `--aws-sigv4`. This keeps the change zero-install. Credentials are fed to curl through a config file on stdin rather than argv, so they never appear in the process list. ### Behaviour - Object layout is identical to S3 (`gitea-runner/<version>/<artifact>`), so migrating consumers later means changing only the host, not the path. - Nightly gets R2 too, matching the existing nightly-to-S3 behaviour. - Missing R2 configuration fails the build. Because custom publishers run as the very last step of the publish pipeline, a preflight `--check-config` step runs right after checkout so the failure happens before anything is built or published rather than after the release already exists. - Verified against a local MinIO instance (site region `auto`, matching R2): successful upload byte-compared after download, plus the missing-variable, wrong-credentials, missing-file and bad-argument paths. ### Required repository secrets These must be configured before the next release run, otherwise the new preflight step will fail the workflow: - `R2_ENDPOINT` (full base URL, e.g. `https://<account>.r2.cloudflarestorage.com`) - `R2_BUCKET` - `R2_ACCESS_KEY_ID` - `R2_SECRET_ACCESS_KEY` ### Known, deliberate wart The publisher runs 109 times for 73 distinct object keys: goreleaser's release pipe already registers `release.extra_files` as `UploadableFile` artifacts, and `internal/exec` appends the publisher's own `extra_files` on top with no de-duplication. It cannot be globbed away because `gobwas/glob` has no substring-exclusion matcher, so `./**.sha256` cannot be narrowed to exclude `*.xz.sha256`. It is harmless since PUT is idempotent, and the redundant `./**.xz` glob is kept on purpose so the publisher declares its own complete file set instead of implicitly depending on the `release:` block's globs. This is documented in a comment above the block. Reviewed-on: https://gitea.com/gitea/runner/pulls/1114 Reviewed-by: Zettat123 <39446+zettat123@noreply.gitea.com>
This commit is contained in:
@@ -93,6 +93,37 @@ blobs:
|
||||
- glob: ./**.xz
|
||||
- glob: ./**.sha256
|
||||
|
||||
# Mirrors the S3 `blobs:` upload above into Cloudflare R2 during the
|
||||
# parallel S3+R2 period (S3 will be removed once migration completes).
|
||||
# A second `blobs:` entry is impossible here since the blob pipe
|
||||
# authenticates from the global AWS_* env with no per-entry
|
||||
# credentials; `publishers:` supports per-entry `env:` instead, so
|
||||
# it's used to invoke scripts/upload-r2.sh once per artifact. Custom
|
||||
# publishers inherit almost nothing from the environment, hence the
|
||||
# explicit R2_* forwarding below.
|
||||
#
|
||||
# This publisher fires 109 times for 73 distinct keys because
|
||||
# goreleaser's release pipe already registers `release.extra_files`
|
||||
# as UploadableFile artifacts, and `internal/exec`'s filterArtifacts
|
||||
# appends this block's own extra_files with no de-duplication. It
|
||||
# can't be globbed away, since gobwas/glob (via goreleaser/fileglob)
|
||||
# has no substring-exclusion matcher. It's harmless: PUT is
|
||||
# idempotent, and the `./**.xz` glob below is kept deliberately so
|
||||
# this publisher declares its own complete file set rather than
|
||||
# implicitly depending on the `release:` block's globs.
|
||||
publishers:
|
||||
- name: cloudflare-r2
|
||||
checksum: true
|
||||
extra_files:
|
||||
- glob: ./**.xz
|
||||
- glob: ./**.sha256
|
||||
cmd: sh scripts/upload-r2.sh {{ abs .ArtifactPath }} gitea-runner/{{ .Version }}/{{ .ArtifactName }}
|
||||
env:
|
||||
- R2_ENDPOINT={{ index .Env "R2_ENDPOINT" }}
|
||||
- R2_BUCKET={{ index .Env "R2_BUCKET" }}
|
||||
- R2_ACCESS_KEY_ID={{ index .Env "R2_ACCESS_KEY_ID" }}
|
||||
- R2_SECRET_ACCESS_KEY={{ index .Env "R2_SECRET_ACCESS_KEY" }}
|
||||
|
||||
archives:
|
||||
- format: binary
|
||||
name_template: "{{ .Binary }}"
|
||||
|
||||
Reference in New Issue
Block a user