Hup failed update in 50%

[georgiats]

Here is the relevant ticket https://jel.ly.fish/1cbfcf44-c186-4590-966f-b1562eab9e61

Application url: https://dashboard.balena-cloud.com/apps/1558934
Device URL: https://dashboard.balena-cloud.com/devices/f97f32106472bd0c442abddc8c877f5b
Device type: UP Board
OS version: balenaOS 2.29.2+rev1
Supervisor version: 9.0.1

The healthchecks show
check_networking failed -->
Some networking issues detected: test_dockerhub: Could not query Docker Hub test_balena_registry: Could not communicate with registry2.balena-cloud.com for authentication

and check_service_restarts failed -->
Some services are restarting unexpectedly: (service: /nodered_2511454_1464029 restart count: 2) (service: /firewallmanager_2511458_1464029 restart count: 1)

I've tried to get the Device diagnostics but I get the error ssh client socket error while initiating SSH connection
Other logs :

-- Logs begin at Mon 2020-08-17 07:01:02 UTC, end at Mon 2020-08-17 11:55:39 UTC. --
Aug 17 10:48:03 f97f321 balenad[750]: time="2020-08-17T10:48:03.726699077Z" level=warning msg="failed to retrieve balena-engine-init version: exec: "balena-engine-init": executable file not found in $PATH"
Aug 17 10:48:47 f97f321 balenad[750]: [124B blob data]
Aug 17 10:48:47 f97f321 balenad[750]: [124B blob data]
Aug 17 10:48:47 f97f321 balenad[750]: [101B blob data]
Aug 17 10:48:47 f97f321 balenad[750]: [106B blob data]
Aug 17 10:48:48 f97f321 balenad[750]: [83B blob data]
Aug 17 10:48:48 f97f321 balenad[750]: [60B blob data]
Aug 17 10:48:48 f97f321 balenad[750]: [75B blob data]
Aug 17 10:48:48 f97f321 balenad[750]: [58B blob data]
Aug 17 10:48:48 f97f321 balenad[750]: [85B blob data]
Aug 17 10:49:00 f97f321 balenad[750]: [51B blob data]
Aug 17 10:49:01 f97f321 balenad[750]: [53B blob data]
Aug 17 10:49:33 f97f321 balenad[750]: time="2020-08-17T10:49:33.844358805Z" level=warning msg="failed to retrieve balena-engine-init version: exec: "balena-engine-init": executable file not found in $PATH"


root@f97f321:~# **tail /mnt/data/balenahup/*log**
==> /mnt/data/balenahup/upgrade-2.x.sh.20200817_064400.log <==
[000000103][LOG]No device-specific pre-update fix for up-board
[000000103][LOG]Starting hostapp-update
delta-818a7f993e42ce3c: Pulling from v2/d982ca1aaa5f4329fb21be088543b487
error pulling image configuration: Get https://registry2.balena-cloud.com/v2/v2/d982ca1aaa5f4329fb21be088543b487/blobs/sha256:a6c6b3eaa2dbc8006b8dbba296e376f006de999438107bfab8cd561814bab595: net/http: TLS handshake timeout
[000000117][LOG]Image type delta, location 'registry2.balena-cloud.com/v2/d982ca1aaa5f4329fb21be088543b487:delta-818a7f993e42ce3c' failed or not found, trying another source
[000000117][LOG]No device-specific pre-update fix for up-board
[000000117][LOG]Starting hostapp-update
sha256:7445bb58d8c338225af69f59879551701e8ecfb8fec1d23d55ef80e94de68617: Pulling from v2/d982ca1aaa5f4329fb21be088543b487
2089148beb07: Pulling fs layer
2089148beb07: Ready to download

==> /mnt/data/balenahup/upgrade-2.x.sh.20200817_073229.log <==
[000000018][LOG]hostapp-update command exists, use that for update
[000000033][LOG]No device-specific pre-update fix for up-board
[000000033][LOG]Starting hostapp-update
delta-818a7f993e42ce3c: Pulling from v2/d982ca1aaa5f4329fb21be088543b487
df517a1b1ab8: Pulling fs layer
df517a1b1ab8: Ready to download
[000001401][LOG]Image type delta, location 'registry2.balena-cloud.com/v2/d982ca1aaa5f4329fb21be088543b487:delta-818a7f993e42ce3c' failed or not found, trying another source
[000001401][LOG]No device-specific pre-update fix for up-board
[000001401][LOG]Starting hostapp-update
[000001451][LOG]Image type balena_registry, location 'registry2.balena-cloud.com/v2/d982ca1aaa5f4329fb21be088543b487@sha256:7445bb58d8c338225af69f59879551701e8ecfb8fec1d23d55ef80e94de68617' failed or not found, trying another source

root@f97f321:~# grep MB /mnt/data/balenahup/*log /mnt/data/balenahup/upgrade-2.x.sh.20200817_064400.log:[000000083][LOG]Found delta image: registry2.balena-cloud.com/v2/d982ca1aaa5f4329fb21be088543b487:delta-818a7f993e42ce3c, size: unknown MB /mnt/data/balenahup/upgrade-2.x.sh.20200817_073229.log:[000000018][LOG]Found delta image: registry2.balena-cloud.com/v2/d982ca1aaa5f4329fb21be088543b487:delta-818a7f993e42ce3c, size: 207 MB```

[jellyfish-bot]

[georgiats] This issue has attached support thread https://jel.ly.fish/1cbfcf44-c186-4590-966f-b1562eab9e61

[jellyfish-bot]

[ntzovanis] This issue has attached support thread https://jel.ly.fish/2a564ae4-2bec-4a6d-b58c-3e67c55c6d3f

[ntzovanis]

I've just linked a new case here. It seems in both cases HUP wasn't running anymore, but the dashboard said updating stuck at 50%. We could do a better job at showing that it failed (and why).

[jellyfish-bot]

[vipulgupta2048] This has attached https://jel.ly.fish/74365c66-f587-44cc-8060-0759c1ec27ab