1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
|
##############################################################################
# Copyright (c) 2026 Eclipse ThreadX contributors
#
# This program and the accompanying materials are made available under the
# terms of the MIT License which is available at
# https://opensource.org/licenses/MIT.
#
# AI Disclosure: This file was largely AI-generated by Claude Code (Opus 5).
# The AI-generated portions may be considered public domain (CC0-1.0)
# and not subject to the project's licence. The human contributor has
# reviewed and verified that the code is correct.
#
# SPDX-License-Identifier: MIT and CC0-1.0
##############################################################################
# Network helpers shared by the CI install scripts. Sourced, not executed --
# there is no shebang, and nothing happens here beyond defining APT_OPTIONS,
# TIMEOUT, TIMEOUT_LONG, retry and fetch.
#
# They lived in install.sh until the RISC-V suite was enabled in CI, at which
# point a second install script was on every pull request's critical path with
# none of them. The lessons below were paid for once; a script that reaches the
# network in this project should not have to learn them again.
#
# The two callers differ in one way worth knowing: install_riscv.sh runs under
# set -e and install.sh does not. That is why install.sh spells out `|| exit 1`
# on the calls that must stop it -- without that, a failed fetch there would be
# carried forward into a build that then failed somewhere less obvious.
# Everything the install scripts do reaches the network, and on this runner pool
# that is not dependable. apt-get update stalled seven times in a single day,
# once for more than two hours, each time with the Azure mirror returning
# nothing and the fallback to archive.ubuntu.com then going silent. Nothing
# bounded a fetch and nothing retried one, so a mirror being down cost a whole
# run rather than a few seconds.
#
# The Acquire options make a slow mirror give up sooner. The retry loop below
# covers a mirror that is down rather than merely slow.
APT_OPTIONS=(-o Acquire::Retries=3
-o Acquire::http::Timeout=20
-o Acquire::https::Timeout=20)
# Two minutes per attempt, killed outright if it ignores the first signal. Three
# attempts plus backoff bounds a command at about six and a half minutes.
#
# Each command is wrapped in timeout rather than left to bound itself. apt's own
# Acquire timeouts were tried first and did not help: a run still sat inside a
# single apt-get update for nine and a half minutes without producing a line,
# having got as far as fetching noble-security InRelease, so the retry loop never
# got a turn and the step timeout was what eventually killed it. Whatever apt is
# waiting on there, it is not something Acquire::http::Timeout covers. timeout
# does not care where the wait is.
#
# timeout goes under sudo, not over it, so that it signals apt itself. Signalling
# sudo instead risks the kill landing on sudo while apt carries on holding the
# dpkg lock, which would leave every retry failing for a different reason than
# the one being retried.
TIMEOUT=(timeout --kill-after=10 120)
# Three minutes, for a single large download rather than a package operation.
# The two RISC-V toolchain tarballs are about 500 MB each and took 39 seconds
# apiece on 10 Sep 2026, so this is a margin of roughly four and a half.
#
# It is deliberately not larger. The install step in regression_template.yml is
# capped at ten minutes, and a per-attempt timeout long enough to swallow that
# cap would leave the retry loop below with no turn to take -- which is the
# exact failure the TIMEOUT comment above records apt producing. Three attempts
# at three minutes plus backoff still does not fit inside ten, so the step
# timeout remains the outer backstop for a server that is genuinely down; what
# the retries buy is recovery from the transient case, which is the common one,
# and a log that says which attempt failed rather than a bare cancelled step.
TIMEOUT_LONG=(timeout --kill-after=10 180)
retry() {
local attempt
for attempt in 1 2 3; do
if "$@"; then
return 0
fi
echo "tx_ci_common: '$*' failed or timed out on attempt ${attempt}"
sleep $((attempt * 10))
done
echo "tx_ci_common: '$*' failed after 3 attempts"
return 1
}
# fetch <url> <destination> <sha256>
#
# Downloads with retries and verifies the digest before the caller is allowed to
# unpack anything. --tries=1 hands retrying to the loop above rather than letting
# wget retry inside a single timeout window and burn it.
#
# The digest is not an independent trust root: it is checked against bytes from
# the same host that publishes the expected value, so it does not prove the
# release was not tampered with at source. What it does buy is that the bytes
# are pinned. A tag can be deleted and re-pushed and an asset can be replaced,
# and today either would be picked up silently; with this, the build stops and
# says which file failed and what it got.
fetch() {
local url=$1
local dest=$2
local sha=$3
retry "${TIMEOUT_LONG[@]}" wget --no-verbose --tries=1 "$url" -O "$dest" || return 1
if ! echo "${sha} ${dest}" | sha256sum --check --status; then
echo "tx_ci_common: checksum mismatch for ${url}" >&2
echo "tx_ci_common: expected ${sha}" >&2
echo "tx_ci_common: actual $(sha256sum "$dest" | cut -d' ' -f1)" >&2
rm -f "$dest"
return 1
fi
}
|
