Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
27 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXX.XX.X..X.............................XXXXXXXX] duration:
30.001119067s
FAIL (2025-08-07 22:26:45 UTC)
operation: sync2.Copy uplink: encryption: context deadline exceeded
Duration: 30.001145541s
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
21 out of 60 is failed (threshold=3) [....X.X.................................XXXXXXXXXXXXXXXXXXX.] duration:
576.648015ms
FAIL (2025-08-07 22:26:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 576.67073ms
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
3 out of 60 is failed (threshold=3) [................................................X..XX.......] duration:
45.106717ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-07 22:26:17 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 45.133202ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
26 out of 60 is failed (threshold=3) [XXXX.XX.X..X..X..........................XXXXXXXXXXXXXXXXX..] duration:
307.98541ms
FAIL (2025-08-07 22:26:05 UTC)
operation: Commit uplink: metaclient: context deadline exceeded
Duration: 308.013875ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
23 out of 60 is failed (threshold=3) [XXXXXXXXXX.X.................................XXXXXXXXXXX.X..] duration:
286.860943ms
FAIL (2025-08-07 22:26:03 UTC)
operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1527233 (12h35m13.476629807s) [upload=12995]: Uploading segments... Aug 7 22:14:38.961574411 (12h35m13.476676997s) [upload=12995]: Got next segment. Inline: true Aug 7 22:14:38.961580546 (12h35m13.47668312s) [upload=12995]: Waiting for error group managing segments... Aug 7 22:14:38.961644878 (12h35m13.476747457s) [upload=12995]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:14:38.961650553 (12h35m13.476753133s) [upload=12995]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:14:38.961655154 (12h35m13.476757734s) [upload=12995]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:15:38.926876179 (12h36m13.441978762s) [upload=12995]: Done uploading segments... Aug 7 22:15:38.926898754 (12h36m13.442001334s) [upload=12995]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:15:39.023006560 (12h36m13.538109126s) [upload=13009]: Starting upload Aug 7 22:15:39.023159911 (12h36m13.538262487s) [upload=13009]: Done starting upload Aug 7 22:15:39.023273936 (12h36m13.538376511s) [upload=13009]: Uploading segments... Aug 7 22:15:39.023358183 (12h36m13.538460767s) [upload=13009]: Got next segment. Inline: true Aug 7 22:15:39.023363552 (12h36m13.538466129s) [upload=13009]: Waiting for error group managing segments... Aug 7 22:15:39.023458469 (12h36m13.538561049s) [upload=13009]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:15:39.023464439 (12h36m13.538567019s) [upload=13009]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:15:39.023491067 (12h36m13.53859365s) [upload=13009]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:16:38.991189889 (12h37m13.506292477s) [upload=13009]: Done uploading segments... Aug 7 22:16:38.991214272 (12h37m13.506316848s) [upload=13009]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:16:39.116496595 (12h37m13.631599182s) [upload=13020]: Starting upload Aug 7 22:16:39.116578790 (12h37m13.631681366s) [upload=13020]: Done starting upload Aug 7 22:16:39.116734512 (12h37m13.631837083s) [upload=13020]: Uploading segments... Aug 7 22:16:39.116797388 (12h37m13.631899986s) [upload=13020]: Got next segment. Inline: true Aug 7 22:16:39.116803839 (12h37m13.631906421s) [upload=13020]: Waiting for error group managing segments... Aug 7 22:16:39.116855539 (12h37m13.631958104s) [upload=13020]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:16:39.116860739 (12h37m13.631963318s) [upload=13020]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:16:39.116864309 (12h37m13.631966875s) [upload=13020]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:17:39.087205419 (12h38m13.602307996s) [upload=13020]: Done uploading segments... Aug 7 22:17:39.087240451 (12h38m13.602343028s) [upload=13020]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:17:39.195279517 (12h38m13.710382082s) [upload=13033]: Starting upload Aug 7 22:17:39.195345128 (12h38m13.710447708s) [upload=13033]: Done starting upload Aug 7 22:17:39.195408125 (12h38m13.710510701s) [upload=13033]: Uploading segments... Aug 7 22:17:39.195557815 (12h38m13.710660402s) [upload=13033]: Got next segment. Inline: true Aug 7 22:17:39.195566565 (12h38m13.710669149s) [upload=13033]: Waiting for error group managing segments... Aug 7 22:17:39.195637820 (12h38m13.710740397s) [upload=13033]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:17:39.195643205 (12h38m13.710745782s) [upload=13033]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:39.195646438 (12h38m13.710749014s) [upload=13033]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:18:39.157680356 (12h39m13.672782934s) [upload=13033]: Done uploading segments... Aug 7 22:18:39.157703138 (12h39m13.672805715s) [upload=13033]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:18:39.252626432 (12h39m13.767729009s) [upload=13048]: Starting upload Aug 7 22:18:39.252783205 (12h39m13.767885845s) [upload=13048]: Done starting upload Aug 7 22:18:39.252852957 (12h39m13.767955535s) [upload=13048]: Uploading segments... Aug 7 22:18:39.252949510 (12h39m13.768052096s) [upload=13048]: Got next segment. Inline: true Aug 7 22:18:39.252955069 (12h39m13.768057637s) [upload=13048]: Waiting for error group managing segments... Aug 7 22:18:39.253031088 (12h39m13.76813365s) [upload=13048]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:18:39.253036915 (12h39m13.768139478s) [upload=13048]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:18:39.253040017 (12h39m13.768142595s) [upload=13048]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:19:39.223199412 (12h40m13.738302239s) [upload=13048]: Done uploading segments... Aug 7 22:19:39.223230125 (12h40m13.738332769s) [upload=13048]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:19:39.328604668 (12h40m13.843707247s) [upload=13065]: Starting upload Aug 7 22:19:39.328870365 (12h40m13.843972956s) [upload=13065]: Done starting upload Aug 7 22:19:39.329172418 (12h40m13.844274986s) [upload=13065]: Uploading segments... Aug 7 22:19:39.329244607 (12h40m13.844347193s) [upload=13065]: Got next segment. Inline: true Aug 7 22:19:39.329250690 (12h40m13.844353268s) [upload=13065]: Waiting for error group managing segments... Aug 7 22:19:39.329302352 (12h40m13.844404927s) [upload=13065]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:19:39.329309321 (12h40m13.844411902s) [upload=13065]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:19:39.329313964 (12h40m13.844416551s) [upload=13065]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:20:39.300343613 (12h41m13.81544618s) [upload=13065]: Done uploading segments... Aug 7 22:20:39.300363356 (12h41m13.815465922s) [upload=13065]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:45.259131358 (12h41m19.774233925s) [upload=13078]: Starting upload Aug 7 22:20:45.259192437 (12h41m19.774295013s) [upload=13078]: Done starting upload Aug 7 22:20:45.259251322 (12h41m19.774353897s) [upload=13078]: Uploading segments... Aug 7 22:20:45.259289829 (12h41m19.774392414s) [upload=13078]: Got next segment. Inline: true Aug 7 22:20:45.259294478 (12h41m19.774397054s) [upload=13078]: Waiting for error group managing segments... Aug 7 22:20:45.259332525 (12h41m19.774435101s) [upload=13078]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:20:45.259337069 (12h41m19.774439635s) [upload=13078]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:45.259340263 (12h41m19.77444284s) [upload=13078]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:21:45.229308083 (12h42m19.744410669s) [upload=13078]: Done uploading segments... Aug 7 22:21:45.229380638 (12h42m19.744483223s) [upload=13078]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:21:59.915154556 (12h42m34.430257139s) [upload=13097]: Starting upload Aug 7 22:21:59.915242531 (12h42m34.430345108s) [upload=13097]: Done starting upload Aug 7 22:21:59.915358515 (12h42m34.430461092s) [upload=13097]: Uploading segments... Aug 7 22:21:59.915398067 (12h42m34.430500653s) [upload=13097]: Got next segment. Inline: true Aug 7 22:21:59.915403151 (12h42m34.430505727s) [upload=13097]: Waiting for error group managing segments... Aug 7 22:21:59.915458296 (12h42m34.430560863s) [upload=13097]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:21:59.915463422 (12h42m34.430565998s) [upload=13097]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:21:59.915466628 (12h42m34.430569269s) [upload=13097]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:22:59.883702618 (12h43m34.398805207s) [upload=13097]: Done uploading segments... Aug 7 22:22:59.883744759 (12h43m34.398847337s) [upload=13097]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:00.217702003 (12h43m34.732804585s) [upload=13110]: Starting upload Aug 7 22:23:00.217805899 (12h43m34.732908483s) [upload=13110]: Done starting upload Aug 7 22:23:00.217896494 (12h43m34.732999072s) [upload=13110]: Uploading segments... Aug 7 22:23:00.217970831 (12h43m34.733073418s) [upload=13110]: Got next segment. Inline: true Aug 7 22:23:00.217979414 (12h43m34.733081991s) [upload=13110]: Waiting for error group managing segments... Aug 7 22:23:00.218043817 (12h43m34.733146394s) [upload=13110]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:23:00.218051159 (12h43m34.733153734s) [upload=13110]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:00.218054412 (12h43m34.733156988s) [upload=13110]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:23:00.464344185 (12h43m34.979446778s) [upload=13110]: Done uploading segments... Aug 7 22:23:00.464376970 (12h43m34.979479551s) [upload=13110]: Upload finished. err: <nil> Aug 7 22:24:07.143696312 (12h44m41.658798899s) [upload=13125]: Starting upload Aug 7 22:24:07.143811006 (12h44m41.658913589s) [upload=13125]: Done starting upload Aug 7 22:24:07.143914011 (12h44m41.659016577s) [upload=13125]: Uploading segments... Aug 7 22:24:07.143973667 (12h44m41.659076252s) [upload=13125]: Got next segment. Inline: true Aug 7 22:24:07.143979447 (12h44m41.659082034s) [upload=13125]: Waiting for error group managing segments... Aug 7 22:24:07.144045718 (12h44m41.659148297s) [upload=13125]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:24:07.144051783 (12h44m41.659154361s) [upload=13125]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:07.144057848 (12h44m41.659160429s) [upload=13125]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:25:07.110905816 (12h45m41.626008396s) [upload=13125]: Done uploading segments... Aug 7 22:25:07.110929433 (12h45m41.626032014s) [upload=13125]: Upload finished. err: metaclient: context deadline exceeded
Duration: 286.890107ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [X.........................................X.................] duration:
1.017491437s
FAIL (2025-08-07 22:26:01 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 70b8ffb330ca946ca16cbba98295b25e9d73c23ceaf12561cb0410c721509eb0 != before 7a7c50f9088555b2e513a4e1d17c684f3e108f16da50d5cb1fb6d77de72f8388
======LOGS======
oad finished. err: <nil>
Aug 7 22:12:09.370677362 (12h32m43.885779928s) [upload=12956]: Starting upload
Aug 7 22:12:09.370743926 (12h32m43.885846503s) [upload=12956]: Done starting upload
Aug 7 22:12:09.371065555 (12h32m43.886168183s) [upload=12956]: Uploading segments...
Aug 7 22:12:09.371127350 (12h32m43.886229927s) [upload=12956]: Got next segment. Inline: false
Aug 7 22:12:09.371179942 (12h32m43.886282519s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 22:12:09.371184578 (12h32m43.886287156s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 22:12:09.514484447 (12h32m44.029587032s) [seg_pos={0 0}, upload=12956]: Begin upload segment...
Aug 7 22:12:09.514553324 (12h32m44.029655914s) [seg_pos={0 0}, upload=12956]: Done begin upload segment.
Aug 7 22:12:09.514604463 (12h32m44.029707051s) [noise=true , address=64.49.13.138:4490 , piece_id=RT5AQYMACT6OGCNBPVOJOLYGYQHHND6TVS3FEE52MXZWXZJLCCVA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.514656287 (12h32m44.029758882s) [noise=true , address=104.36.82.34:4575 , piece_id=MQE24KHKGXFJPHPKZH4NAI3G2FSOWA42JGN2WEXU47J6ALXEMFOQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.515221313 (12h32m44.030323899s) [noise=true , address=104.36.82.44:4873 , piece_id=GLETUFDVURZ3XPA2OHFTOA7NLKIQSPKF6HVFNPGBOOPU7ICLEOAA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.515435936 (12h32m44.030538517s) [noise=true , address=104.36.82.59:4326 , piece_id=SHCRSGVR3LRTNV2GEXSWQWSOYPHUOOHDJPYOUALO4M34TZGCHHBA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.515609669 (12h32m44.030712245s) [noise=true , address=79.127.201.215:4357 , piece_id=I2SQGM7NHIUHKNQG6RT3PH3W6P5OKJPP3MFJ65QC7CTN36DUYYJA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516008521 (12h32m44.03111111s) [noise=true , address=104.36.82.56:4257 , piece_id=6CTH2UDV6ECR5GOBNLG36WD63IHMQTVYH3IUYH33RZYB4ZQ6LQ2A, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516150437 (12h32m44.031253023s) [noise=true , address=45.91.116.11:4132 , piece_id=7DYDCKW5NLR22DRUCWIMELTSHM325ESELDERG26BTA2YLWQE2HHA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516344626 (12h32m44.031447208s) [noise=true , address=79.127.205.235:4670 , piece_id=3GWZPOLPZFZUAJKZ6MOXRTGGE7YDBMNP53WLJXEVOYW2ES5MLZPA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516841797 (12h32m44.031944374s) [noise=true , address=64.49.13.154:4934 , piece_id=5T4YXQHU4DJEZCEAHQT3U4P636W5WDGSQCVCB73XQOCJFAV4O2MA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517033011 (12h32m44.032135594s) [noise=true , address=104.36.82.14:4248 , piece_id=7ZRBZNDCIDWWONITBOTR3OKLOWVCZ5VCANY6YAQQW6JYSUBUMJZA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517240434 (12h32m44.032343025s) [noise=true , address=104.36.82.58:4292 , piece_id=PFHGP3WHANRR4CSDD3VN5MBXQ65OV7UGJ5WDUP34AZYQZAMRLU7A, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517628325 (12h32m44.032730907s) [noise=true , address=64.49.13.137:4493 , piece_id=432XMLABWE3DSWXU3B3BTREMG6CP7ILTNU4RJPA6JLHEJNX5POIA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517733543 (12h32m44.032836126s) [noise=true , address=104.36.82.61:4386 , piece_id=PNWQU2MPHMRVDXWMMWVNLWGLFQAMEJRLGJXFRCT4M3QYDEG5GSFQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.518003213 (12h32m44.033105792s) [noise=true , address=104.36.82.5:4844 , piece_id=GDXNIVT4KT5XFLFAR7B7NMHQDCE2OGGJQYHWID5ASAWO2IVNBN7Q, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.518348003 (12h32m44.03345058s) [noise=true , address=77.90.31.14:4683 , piece_id=4NNJC6OFHOZNQIKLWRGP7JOCMJAF5NHGTF24A6TID7JVKXNFZATQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.518755006 (12h32m44.033857585s) [noise=true , address=104.36.82.15:4267 , piece_id=DSRR5EHTM75MM7P4MG2YNPHWAE7WHALDCRE4L3YZKKH2BMKRTKAA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.522094892 (12h32m44.037197472s) [noise=true , address=104.36.82.18:4383 , piece_id=XVON7B5A6KIUDIZUJKNMNDQBQ5JNTCORJO7JL4NLPYOIGYU63TXQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.523039320 (12h32m44.038141897s) [noise=true , address=104.36.82.43:4856 , piece_id=PIKKNIOZICWM7EJM3NSRIJWPLBXIAP7WNT7RBREAHU6JRXZDCHYA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.590300484 (12h32m44.105403049s) [upload=12956]: Next returned nil segment
Aug 7 22:12:09.590317010 (12h32m44.105419591s) [upload=12956]: Waiting for error group managing segments...
Aug 7 22:12:09.604186438 (12h32m44.119289008s) [noise=true , address=45.91.116.11:4132 , piece_id=7DYDCKW5NLR22DRUCWIMELTSHM325ESELDERG26BTA2YLWQE2HHA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.606286681 (12h32m44.121389287s) [noise=true , address=104.36.82.59:4326 , piece_id=SHCRSGVR3LRTNV2GEXSWQWSOYPHUOOHDJPYOUALO4M34TZGCHHBA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.620487996 (12h32m44.135590569s) [noise=true , address=104.36.82.61:4386 , piece_id=PNWQU2MPHMRVDXWMMWVNLWGLFQAMEJRLGJXFRCT4M3QYDEG5GSFQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.620594571 (12h32m44.135697142s) [noise=true , address=77.90.31.14:4683 , piece_id=4NNJC6OFHOZNQIKLWRGP7JOCMJAF5NHGTF24A6TID7JVKXNFZATQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.625884604 (12h32m44.140987167s) [noise=true , address=104.36.82.15:4267 , piece_id=DSRR5EHTM75MM7P4MG2YNPHWAE7WHALDCRE4L3YZKKH2BMKRTKAA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.626396996 (12h32m44.141499578s) [noise=true , address=104.36.82.58:4292 , piece_id=PFHGP3WHANRR4CSDD3VN5MBXQ65OV7UGJ5WDUP34AZYQZAMRLU7A, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.629464012 (12h32m44.144566574s) [noise=true , address=104.36.82.5:4844 , piece_id=GDXNIVT4KT5XFLFAR7B7NMHQDCE2OGGJQYHWID5ASAWO2IVNBN7Q, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.643428107 (12h32m44.158530675s) [noise=true , address=104.36.82.14:4248 , piece_id=7ZRBZNDCIDWWONITBOTR3OKLOWVCZ5VCANY6YAQQW6JYSUBUMJZA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.643719221 (12h32m44.15882179s) [noise=true , address=104.36.82.44:4873 , piece_id=GLETUFDVURZ3XPA2OHFTOA7NLKIQSPKF6HVFNPGBOOPU7ICLEOAA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.644290441 (12h32m44.159393003s) [noise=true , address=104.36.82.34:4575 , piece_id=MQE24KHKGXFJPHPKZH4NAI3G2FSOWA42JGN2WEXU47J6ALXEMFOQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.646798488 (12h32m44.161901065s) [noise=true , address=104.36.82.43:4856 , piece_id=PIKKNIOZICWM7EJM3NSRIJWPLBXIAP7WNT7RBREAHU6JRXZDCHYA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.656991689 (12h32m44.172094259s) [noise=true , address=104.36.82.56:4257 , piece_id=6CTH2UDV6ECR5GOBNLG36WD63IHMQTVYH3IUYH33RZYB4ZQ6LQ2A, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.835525769 (12h32m44.350628346s) [noise=true , address=79.127.201.215:4357 , piece_id=I2SQGM7NHIUHKNQG6RT3PH3W6P5OKJPP3MFJ65QC7CTN36DUYYJA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:10.167762322 (12h32m44.682864893s) [noise=true , address=64.49.13.154:4934 , piece_id=5T4YXQHU4DJEZCEAHQT3U4P636W5WDGSQCVCB73XQOCJFAV4O2MA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:10.167785966 (12h32m44.682888542s) [seg_pos={0 0}, upload=12956]: Segment reached optimal threshold of 14 pieces.
Aug 7 22:12:10.167971590 (12h32m44.683074167s) [noise=true , address=64.49.13.137:4493 , piece_id=432XMLABWE3DSWXU3B3BTREMG6CP7ILTNU4RJPA6JLHEJNX5POIA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12N91KV4t7yYJvKLFMaXHjByCZpsNgWFnYyZHJE7BqEYBGMLAi9): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168073316 (12h32m44.683175878s) [noise=true , address=104.36.82.18:4383 , piece_id=XVON7B5A6KIUDIZUJKNMNDQBQ5JNTCORJO7JL4NLPYOIGYU63TXQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Wtu7qHz3z5gYVPLJzaKKH3orBrtBUEVUeHRUaUe4xJGZv4u4z): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168146785 (12h32m44.683249347s) [noise=true , address=64.49.13.138:4490 , piece_id=RT5AQYMACT6OGCNBPVOJOLYGYQHHND6TVS3FEE52MXZWXZJLCCVA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Jp1vcm8dpppQHEfAtYmM29CDS3mqhojhuVrzkL3PsYHW2q8zA): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168258506 (12h32m44.683361075s) [noise=true , address=79.127.205.235:4670 , piece_id=3GWZPOLPZFZUAJKZ6MOXRTGGE7YDBMNP53WLJXEVOYW2ES5MLZPA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JgTfMzsd3rMqQwNim6BoZSiePoLW3gcUtraj84Kn4Szy7aNga): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168271822 (12h32m44.68337439s) [seg_pos={0 0}, upload=12956]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 22:12:10.168367951 (12h32m44.683470527s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 22:12:10.168372753 (12h32m44.683475329s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 22:12:11.355196489 (12h32m45.870299059s) [upload=12956]: Done uploading segments...
Aug 7 22:12:11.355214126 (12h32m45.870316692s) [upload=12956]: Upload finished. err: <nil>
Duration: 1.017523099s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
26 out of 60 is failed (threshold=3) [XXXXXXXXX.XX...X.............................XXXXXXXXXXX.XXX] duration:
1m0.001172147s
FAIL (2025-08-07 22:26:01 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.00118793s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
24 out of 60 is failed (threshold=3) [XX....XX................................XXXXXXXXXXXXXXXXXXXX] duration:
30.000777153s
FAIL (2025-08-07 22:26:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000795651s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
29.583683ms remote:
"tcp:136.0.77.2:443" target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-07 22:25:56 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded
Duration: 29.611707ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
4 out of 60 is failed (threshold=3) [X................................................X..XX......] duration:
39.369504ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-07 22:25:55 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 39.395265ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
2m0.001044813s remote:
"tcp:136.0.77.2:443" target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-07 22:25:55 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded
Duration: 2m0.001086157s
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
25 out of 60 is failed (threshold=3) [X.XX...XXX...............................XXXXXXXXXXXXXXXXXXX] duration:
30.000599506s
FAIL (2025-08-07 22:25:54 UTC)
operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== (12h39m7.813236527s) [upload=13046]: Uploading segments... Aug 7 22:18:33.298195648 (12h39m7.813298236s) [upload=13046]: Got next segment. Inline: true Aug 7 22:18:33.298201348 (12h39m7.813303934s) [upload=13046]: Waiting for error group managing segments... Aug 7 22:18:33.298250060 (12h39m7.813352642s) [upload=13046]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:18:33.298255690 (12h39m7.813358273s) [upload=13046]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:18:33.298259809 (12h39m7.813362387s) [upload=13046]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:19:03.266409729 (12h39m37.781512306s) [upload=13046]: Done uploading segments... Aug 7 22:19:03.266458971 (12h39m37.781561549s) [upload=13046]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:19:30.020553557 (12h40m4.535656125s) [upload=13060]: Starting upload Aug 7 22:19:30.020630355 (12h40m4.535732931s) [upload=13060]: Done starting upload Aug 7 22:19:30.020870620 (12h40m4.535973198s) [upload=13060]: Uploading segments... Aug 7 22:19:30.020984540 (12h40m4.536087103s) [upload=13060]: Got next segment. Inline: true Aug 7 22:19:30.020990929 (12h40m4.536093523s) [upload=13060]: Waiting for error group managing segments... Aug 7 22:19:30.021057882 (12h40m4.536160461s) [upload=13060]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:19:30.021063707 (12h40m4.536166287s) [upload=13060]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:19:30.021067070 (12h40m4.536169633s) [upload=13060]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:19:59.990140275 (12h40m34.505242847s) [upload=13060]: Done uploading segments... Aug 7 22:19:59.990166176 (12h40m34.505268763s) [upload=13060]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:14.462825235 (12h40m48.977927811s) [upload=13071]: Starting upload Aug 7 22:20:14.462946370 (12h40m48.978048965s) [upload=13071]: Done starting upload Aug 7 22:20:14.463094242 (12h40m48.978196809s) [upload=13071]: Uploading segments... Aug 7 22:20:14.463162579 (12h40m48.978265164s) [upload=13071]: Got next segment. Inline: true Aug 7 22:20:14.463168150 (12h40m48.978270729s) [upload=13071]: Waiting for error group managing segments... Aug 7 22:20:14.463216881 (12h40m48.978319457s) [upload=13071]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:20:14.463226562 (12h40m48.978329138s) [upload=13071]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:14.463230088 (12h40m48.978332654s) [upload=13071]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:20:44.419309349 (12h41m18.934411917s) [upload=13071]: Done uploading segments... Aug 7 22:20:44.419327995 (12h41m18.934430565s) [upload=13071]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:59.026227228 (12h41m33.541329811s) [upload=13081]: Starting upload Aug 7 22:20:59.026335834 (12h41m33.541438465s) [upload=13081]: Done starting upload Aug 7 22:20:59.026413148 (12h41m33.541515725s) [upload=13081]: Uploading segments... Aug 7 22:20:59.026487957 (12h41m33.541590544s) [upload=13081]: Got next segment. Inline: true Aug 7 22:20:59.026494660 (12h41m33.54159724s) [upload=13081]: Waiting for error group managing segments... Aug 7 22:20:59.026548283 (12h41m33.541650867s) [upload=13081]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:20:59.026554014 (12h41m33.541656591s) [upload=13081]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:59.026557057 (12h41m33.541659621s) [upload=13081]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:21:28.995870666 (12h42m3.510973249s) [upload=13081]: Done uploading segments... Aug 7 22:21:28.995890051 (12h42m3.510992681s) [upload=13081]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:21:29.129769938 (12h42m3.644872515s) [upload=13090]: Starting upload Aug 7 22:21:29.129856429 (12h42m3.644959013s) [upload=13090]: Done starting upload Aug 7 22:21:29.129943698 (12h42m3.645046276s) [upload=13090]: Uploading segments... Aug 7 22:21:29.129985259 (12h42m3.645087845s) [upload=13090]: Got next segment. Inline: true Aug 7 22:21:29.129990476 (12h42m3.645093052s) [upload=13090]: Waiting for error group managing segments... Aug 7 22:21:29.130027566 (12h42m3.645130145s) [upload=13090]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:21:29.130031892 (12h42m3.645134469s) [upload=13090]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:21:29.130035507 (12h42m3.645138082s) [upload=13090]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:21:59.086555047 (12h42m33.601657631s) [upload=13090]: Done uploading segments... Aug 7 22:21:59.086578788 (12h42m33.601681377s) [upload=13090]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:22:21.787894978 (12h42m56.302997579s) [upload=13102]: Starting upload Aug 7 22:22:21.787975806 (12h42m56.303078386s) [upload=13102]: Done starting upload Aug 7 22:22:21.788049720 (12h42m56.303152297s) [upload=13102]: Uploading segments... Aug 7 22:22:21.788092210 (12h42m56.303194795s) [upload=13102]: Got next segment. Inline: true Aug 7 22:22:21.788097559 (12h42m56.303200126s) [upload=13102]: Waiting for error group managing segments... Aug 7 22:22:21.788141372 (12h42m56.303243949s) [upload=13102]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:22:21.788146123 (12h42m56.30324876s) [upload=13102]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:21.788149819 (12h42m56.303252395s) [upload=13102]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:22:51.758466888 (12h43m26.273569473s) [upload=13102]: Done uploading segments... Aug 7 22:22:51.758489416 (12h43m26.273591984s) [upload=13102]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:17.561525115 (12h43m52.076627681s) [upload=13117]: Starting upload Aug 7 22:23:17.561618947 (12h43m52.076721524s) [upload=13117]: Done starting upload Aug 7 22:23:17.561741793 (12h43m52.07684436s) [upload=13117]: Uploading segments... Aug 7 22:23:17.561843225 (12h43m52.076945802s) [upload=13117]: Got next segment. Inline: true Aug 7 22:23:17.561848606 (12h43m52.076951175s) [upload=13117]: Waiting for error group managing segments... Aug 7 22:23:17.561909318 (12h43m52.077011885s) [upload=13117]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:23:17.561914564 (12h43m52.077017202s) [upload=13117]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:17.561918167 (12h43m52.077020738s) [upload=13117]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:23:47.532068483 (12h44m22.047171052s) [upload=13117]: Done uploading segments... Aug 7 22:23:47.532095698 (12h44m22.047198268s) [upload=13117]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:55.295508312 (12h44m29.810610893s) [upload=13123]: Starting upload Aug 7 22:23:55.295607041 (12h44m29.81070962s) [upload=13123]: Done starting upload Aug 7 22:23:55.295735905 (12h44m29.810838474s) [upload=13123]: Uploading segments... Aug 7 22:23:55.295799978 (12h44m29.810902554s) [upload=13123]: Got next segment. Inline: true Aug 7 22:23:55.295805637 (12h44m29.810908214s) [upload=13123]: Waiting for error group managing segments... Aug 7 22:23:55.295849955 (12h44m29.810952532s) [upload=13123]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:23:55.295854812 (12h44m29.810957387s) [upload=13123]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:55.295858141 (12h44m29.810960717s) [upload=13123]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:24:25.257624495 (12h44m59.772727074s) [upload=13123]: Done uploading segments... Aug 7 22:24:25.257647063 (12h44m59.77274964s) [upload=13123]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:24:51.263752764 (12h45m25.778855407s) [upload=13135]: Starting upload Aug 7 22:24:51.263877660 (12h45m25.778980247s) [upload=13135]: Done starting upload Aug 7 22:24:51.263966776 (12h45m25.779069345s) [upload=13135]: Uploading segments... Aug 7 22:24:51.264018780 (12h45m25.779121365s) [upload=13135]: Got next segment. Inline: true Aug 7 22:24:51.264023895 (12h45m25.779126462s) [upload=13135]: Waiting for error group managing segments... Aug 7 22:24:51.264067265 (12h45m25.779169838s) [upload=13135]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:24:51.264097890 (12h45m25.779200467s) [upload=13135]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:51.264101967 (12h45m25.779204534s) [upload=13135]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:25:21.220725953 (12h45m55.735828536s) [upload=13135]: Done uploading segments... Aug 7 22:25:21.220748855 (12h45m55.735851436s) [upload=13135]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:25.019445687 (12h45m59.534548303s) [upload=13145]: Starting upload Aug 7 22:25:25.019581109 (12h45m59.534683692s) [upload=13145]: Done starting upload Aug 7 22:25:25.019703288 (12h45m59.534805856s) [upload=13145]: Uploading segments... Aug 7 22:25:25.019823488 (12h45m59.534926072s) [upload=13145]: Got next segment. Inline: true Aug 7 22:25:25.019831359 (12h45m59.534933942s) [upload=13145]: Waiting for error group managing segments... Aug 7 22:25:25.019906993 (12h45m59.535009557s) [upload=13145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:25:25.019929793 (12h45m59.535032371s) [upload=13145]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:25.019934887 (12h45m59.535037461s) [upload=13145]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:25:54.989474261 (12h46m29.504576827s) [upload=13145]: Done uploading segments... Aug 7 22:25:54.989504165 (12h46m29.50460673s) [upload=13145]: Upload finished. err: metaclient: context deadline exceeded
Duration: 30.000620911s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
27 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXX.XX.X..X.............................XXXXXXX] duration:
30.001005053s
FAIL (2025-08-07 22:25:54 UTC)
operation: sync2.Copy uplink: encryption: context deadline exceeded
Duration: 30.00102171s
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
22 out of 60 is failed (threshold=3) [X....X.X.................................XXXXXXXXXXXXXXXXXXX] duration:
30.000851258s
FAIL (2025-08-07 22:25:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000873747s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
24 out of 60 is failed (threshold=3) [XXX....XX................................XXXXXXXXXXXXXXXXXXX] duration:
30.000618901s
FAIL (2025-08-07 22:25:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000642618s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
25 out of 60 is failed (threshold=3) [XX.XX...XXX...............................XXXXXXXXXXXXXXXXXX] duration:
30.001472898s
FAIL (2025-08-07 22:25:21 UTC)
operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4 (12h38m18.262362591s) [upload=13037]: Uploading segments... Aug 7 22:17:43.747409410 (12h38m18.262511987s) [upload=13037]: Got next segment. Inline: true Aug 7 22:17:43.747414935 (12h38m18.262517512s) [upload=13037]: Waiting for error group managing segments... Aug 7 22:17:43.747457457 (12h38m18.262560036s) [upload=13037]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:17:43.747461997 (12h38m18.262564572s) [upload=13037]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:43.747465159 (12h38m18.262567736s) [upload=13037]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:18:13.715508579 (12h38m48.230611159s) [upload=13037]: Done uploading segments... Aug 7 22:18:13.715531957 (12h38m48.230634536s) [upload=13037]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:18:33.297944141 (12h39m7.813046721s) [upload=13046]: Starting upload Aug 7 22:18:33.298041745 (12h39m7.813144331s) [upload=13046]: Done starting upload Aug 7 22:18:33.298133963 (12h39m7.813236527s) [upload=13046]: Uploading segments... Aug 7 22:18:33.298195648 (12h39m7.813298236s) [upload=13046]: Got next segment. Inline: true Aug 7 22:18:33.298201348 (12h39m7.813303934s) [upload=13046]: Waiting for error group managing segments... Aug 7 22:18:33.298250060 (12h39m7.813352642s) [upload=13046]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:18:33.298255690 (12h39m7.813358273s) [upload=13046]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:18:33.298259809 (12h39m7.813362387s) [upload=13046]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:19:03.266409729 (12h39m37.781512306s) [upload=13046]: Done uploading segments... Aug 7 22:19:03.266458971 (12h39m37.781561549s) [upload=13046]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:19:30.020553557 (12h40m4.535656125s) [upload=13060]: Starting upload Aug 7 22:19:30.020630355 (12h40m4.535732931s) [upload=13060]: Done starting upload Aug 7 22:19:30.020870620 (12h40m4.535973198s) [upload=13060]: Uploading segments... Aug 7 22:19:30.020984540 (12h40m4.536087103s) [upload=13060]: Got next segment. Inline: true Aug 7 22:19:30.020990929 (12h40m4.536093523s) [upload=13060]: Waiting for error group managing segments... Aug 7 22:19:30.021057882 (12h40m4.536160461s) [upload=13060]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:19:30.021063707 (12h40m4.536166287s) [upload=13060]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:19:30.021067070 (12h40m4.536169633s) [upload=13060]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:19:59.990140275 (12h40m34.505242847s) [upload=13060]: Done uploading segments... Aug 7 22:19:59.990166176 (12h40m34.505268763s) [upload=13060]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:14.462825235 (12h40m48.977927811s) [upload=13071]: Starting upload Aug 7 22:20:14.462946370 (12h40m48.978048965s) [upload=13071]: Done starting upload Aug 7 22:20:14.463094242 (12h40m48.978196809s) [upload=13071]: Uploading segments... Aug 7 22:20:14.463162579 (12h40m48.978265164s) [upload=13071]: Got next segment. Inline: true Aug 7 22:20:14.463168150 (12h40m48.978270729s) [upload=13071]: Waiting for error group managing segments... Aug 7 22:20:14.463216881 (12h40m48.978319457s) [upload=13071]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:20:14.463226562 (12h40m48.978329138s) [upload=13071]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:14.463230088 (12h40m48.978332654s) [upload=13071]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:20:44.419309349 (12h41m18.934411917s) [upload=13071]: Done uploading segments... Aug 7 22:20:44.419327995 (12h41m18.934430565s) [upload=13071]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:59.026227228 (12h41m33.541329811s) [upload=13081]: Starting upload Aug 7 22:20:59.026335834 (12h41m33.541438465s) [upload=13081]: Done starting upload Aug 7 22:20:59.026413148 (12h41m33.541515725s) [upload=13081]: Uploading segments... Aug 7 22:20:59.026487957 (12h41m33.541590544s) [upload=13081]: Got next segment. Inline: true Aug 7 22:20:59.026494660 (12h41m33.54159724s) [upload=13081]: Waiting for error group managing segments... Aug 7 22:20:59.026548283 (12h41m33.541650867s) [upload=13081]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:20:59.026554014 (12h41m33.541656591s) [upload=13081]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:59.026557057 (12h41m33.541659621s) [upload=13081]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:21:28.995870666 (12h42m3.510973249s) [upload=13081]: Done uploading segments... Aug 7 22:21:28.995890051 (12h42m3.510992681s) [upload=13081]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:21:29.129769938 (12h42m3.644872515s) [upload=13090]: Starting upload Aug 7 22:21:29.129856429 (12h42m3.644959013s) [upload=13090]: Done starting upload Aug 7 22:21:29.129943698 (12h42m3.645046276s) [upload=13090]: Uploading segments... Aug 7 22:21:29.129985259 (12h42m3.645087845s) [upload=13090]: Got next segment. Inline: true Aug 7 22:21:29.129990476 (12h42m3.645093052s) [upload=13090]: Waiting for error group managing segments... Aug 7 22:21:29.130027566 (12h42m3.645130145s) [upload=13090]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:21:29.130031892 (12h42m3.645134469s) [upload=13090]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:21:29.130035507 (12h42m3.645138082s) [upload=13090]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:21:59.086555047 (12h42m33.601657631s) [upload=13090]: Done uploading segments... Aug 7 22:21:59.086578788 (12h42m33.601681377s) [upload=13090]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:22:21.787894978 (12h42m56.302997579s) [upload=13102]: Starting upload Aug 7 22:22:21.787975806 (12h42m56.303078386s) [upload=13102]: Done starting upload Aug 7 22:22:21.788049720 (12h42m56.303152297s) [upload=13102]: Uploading segments... Aug 7 22:22:21.788092210 (12h42m56.303194795s) [upload=13102]: Got next segment. Inline: true Aug 7 22:22:21.788097559 (12h42m56.303200126s) [upload=13102]: Waiting for error group managing segments... Aug 7 22:22:21.788141372 (12h42m56.303243949s) [upload=13102]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:22:21.788146123 (12h42m56.30324876s) [upload=13102]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:21.788149819 (12h42m56.303252395s) [upload=13102]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:22:51.758466888 (12h43m26.273569473s) [upload=13102]: Done uploading segments... Aug 7 22:22:51.758489416 (12h43m26.273591984s) [upload=13102]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:17.561525115 (12h43m52.076627681s) [upload=13117]: Starting upload Aug 7 22:23:17.561618947 (12h43m52.076721524s) [upload=13117]: Done starting upload Aug 7 22:23:17.561741793 (12h43m52.07684436s) [upload=13117]: Uploading segments... Aug 7 22:23:17.561843225 (12h43m52.076945802s) [upload=13117]: Got next segment. Inline: true Aug 7 22:23:17.561848606 (12h43m52.076951175s) [upload=13117]: Waiting for error group managing segments... Aug 7 22:23:17.561909318 (12h43m52.077011885s) [upload=13117]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:23:17.561914564 (12h43m52.077017202s) [upload=13117]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:17.561918167 (12h43m52.077020738s) [upload=13117]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:23:47.532068483 (12h44m22.047171052s) [upload=13117]: Done uploading segments... Aug 7 22:23:47.532095698 (12h44m22.047198268s) [upload=13117]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:55.295508312 (12h44m29.810610893s) [upload=13123]: Starting upload Aug 7 22:23:55.295607041 (12h44m29.81070962s) [upload=13123]: Done starting upload Aug 7 22:23:55.295735905 (12h44m29.810838474s) [upload=13123]: Uploading segments... Aug 7 22:23:55.295799978 (12h44m29.810902554s) [upload=13123]: Got next segment. Inline: true Aug 7 22:23:55.295805637 (12h44m29.810908214s) [upload=13123]: Waiting for error group managing segments... Aug 7 22:23:55.295849955 (12h44m29.810952532s) [upload=13123]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:23:55.295854812 (12h44m29.810957387s) [upload=13123]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:55.295858141 (12h44m29.810960717s) [upload=13123]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:24:25.257624495 (12h44m59.772727074s) [upload=13123]: Done uploading segments... Aug 7 22:24:25.257647063 (12h44m59.77274964s) [upload=13123]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:24:51.263752764 (12h45m25.778855407s) [upload=13135]: Starting upload Aug 7 22:24:51.263877660 (12h45m25.778980247s) [upload=13135]: Done starting upload Aug 7 22:24:51.263966776 (12h45m25.779069345s) [upload=13135]: Uploading segments... Aug 7 22:24:51.264018780 (12h45m25.779121365s) [upload=13135]: Got next segment. Inline: true Aug 7 22:24:51.264023895 (12h45m25.779126462s) [upload=13135]: Waiting for error group managing segments... Aug 7 22:24:51.264067265 (12h45m25.779169838s) [upload=13135]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:24:51.264097890 (12h45m25.779200467s) [upload=13135]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:51.264101967 (12h45m25.779204534s) [upload=13135]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:25:21.220725953 (12h45m55.735828536s) [upload=13135]: Done uploading segments... Aug 7 22:25:21.220748855 (12h45m55.735851436s) [upload=13135]: Upload finished. err: metaclient: context deadline exceeded
Duration: 30.001499815s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
5 out of 60 is failed (threshold=3) [XX................................................X..XX.....] duration:
57.363118ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-07 22:25:17 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 57.384684ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
27 out of 60 is failed (threshold=3) [XXXXX.XX.X..X..X..........................XXXXXXXXXXXXXXXXX.] duration:
250.02083ms
FAIL (2025-08-07 22:25:16 UTC)
operation: Commit uplink: metaclient: context deadline exceeded
Duration: 250.067418ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.X.........................................X................] duration:
608.556212ms
FAIL (2025-08-07 22:25:11 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 70b8ffb330ca946ca16cbba98295b25e9d73c23ceaf12561cb0410c721509eb0 != before 7a7c50f9088555b2e513a4e1d17c684f3e108f16da50d5cb1fb6d77de72f8388
======LOGS======
oad finished. err: <nil>
Aug 7 22:12:09.370677362 (12h32m43.885779928s) [upload=12956]: Starting upload
Aug 7 22:12:09.370743926 (12h32m43.885846503s) [upload=12956]: Done starting upload
Aug 7 22:12:09.371065555 (12h32m43.886168183s) [upload=12956]: Uploading segments...
Aug 7 22:12:09.371127350 (12h32m43.886229927s) [upload=12956]: Got next segment. Inline: false
Aug 7 22:12:09.371179942 (12h32m43.886282519s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 22:12:09.371184578 (12h32m43.886287156s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 22:12:09.514484447 (12h32m44.029587032s) [seg_pos={0 0}, upload=12956]: Begin upload segment...
Aug 7 22:12:09.514553324 (12h32m44.029655914s) [seg_pos={0 0}, upload=12956]: Done begin upload segment.
Aug 7 22:12:09.514604463 (12h32m44.029707051s) [noise=true , address=64.49.13.138:4490 , piece_id=RT5AQYMACT6OGCNBPVOJOLYGYQHHND6TVS3FEE52MXZWXZJLCCVA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.514656287 (12h32m44.029758882s) [noise=true , address=104.36.82.34:4575 , piece_id=MQE24KHKGXFJPHPKZH4NAI3G2FSOWA42JGN2WEXU47J6ALXEMFOQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.515221313 (12h32m44.030323899s) [noise=true , address=104.36.82.44:4873 , piece_id=GLETUFDVURZ3XPA2OHFTOA7NLKIQSPKF6HVFNPGBOOPU7ICLEOAA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.515435936 (12h32m44.030538517s) [noise=true , address=104.36.82.59:4326 , piece_id=SHCRSGVR3LRTNV2GEXSWQWSOYPHUOOHDJPYOUALO4M34TZGCHHBA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.515609669 (12h32m44.030712245s) [noise=true , address=79.127.201.215:4357 , piece_id=I2SQGM7NHIUHKNQG6RT3PH3W6P5OKJPP3MFJ65QC7CTN36DUYYJA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516008521 (12h32m44.03111111s) [noise=true , address=104.36.82.56:4257 , piece_id=6CTH2UDV6ECR5GOBNLG36WD63IHMQTVYH3IUYH33RZYB4ZQ6LQ2A, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516150437 (12h32m44.031253023s) [noise=true , address=45.91.116.11:4132 , piece_id=7DYDCKW5NLR22DRUCWIMELTSHM325ESELDERG26BTA2YLWQE2HHA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516344626 (12h32m44.031447208s) [noise=true , address=79.127.205.235:4670 , piece_id=3GWZPOLPZFZUAJKZ6MOXRTGGE7YDBMNP53WLJXEVOYW2ES5MLZPA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516841797 (12h32m44.031944374s) [noise=true , address=64.49.13.154:4934 , piece_id=5T4YXQHU4DJEZCEAHQT3U4P636W5WDGSQCVCB73XQOCJFAV4O2MA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517033011 (12h32m44.032135594s) [noise=true , address=104.36.82.14:4248 , piece_id=7ZRBZNDCIDWWONITBOTR3OKLOWVCZ5VCANY6YAQQW6JYSUBUMJZA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517240434 (12h32m44.032343025s) [noise=true , address=104.36.82.58:4292 , piece_id=PFHGP3WHANRR4CSDD3VN5MBXQ65OV7UGJ5WDUP34AZYQZAMRLU7A, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517628325 (12h32m44.032730907s) [noise=true , address=64.49.13.137:4493 , piece_id=432XMLABWE3DSWXU3B3BTREMG6CP7ILTNU4RJPA6JLHEJNX5POIA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517733543 (12h32m44.032836126s) [noise=true , address=104.36.82.61:4386 , piece_id=PNWQU2MPHMRVDXWMMWVNLWGLFQAMEJRLGJXFRCT4M3QYDEG5GSFQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.518003213 (12h32m44.033105792s) [noise=true , address=104.36.82.5:4844 , piece_id=GDXNIVT4KT5XFLFAR7B7NMHQDCE2OGGJQYHWID5ASAWO2IVNBN7Q, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.518348003 (12h32m44.03345058s) [noise=true , address=77.90.31.14:4683 , piece_id=4NNJC6OFHOZNQIKLWRGP7JOCMJAF5NHGTF24A6TID7JVKXNFZATQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.518755006 (12h32m44.033857585s) [noise=true , address=104.36.82.15:4267 , piece_id=DSRR5EHTM75MM7P4MG2YNPHWAE7WHALDCRE4L3YZKKH2BMKRTKAA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.522094892 (12h32m44.037197472s) [noise=true , address=104.36.82.18:4383 , piece_id=XVON7B5A6KIUDIZUJKNMNDQBQ5JNTCORJO7JL4NLPYOIGYU63TXQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.523039320 (12h32m44.038141897s) [noise=true , address=104.36.82.43:4856 , piece_id=PIKKNIOZICWM7EJM3NSRIJWPLBXIAP7WNT7RBREAHU6JRXZDCHYA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.590300484 (12h32m44.105403049s) [upload=12956]: Next returned nil segment
Aug 7 22:12:09.590317010 (12h32m44.105419591s) [upload=12956]: Waiting for error group managing segments...
Aug 7 22:12:09.604186438 (12h32m44.119289008s) [noise=true , address=45.91.116.11:4132 , piece_id=7DYDCKW5NLR22DRUCWIMELTSHM325ESELDERG26BTA2YLWQE2HHA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.606286681 (12h32m44.121389287s) [noise=true , address=104.36.82.59:4326 , piece_id=SHCRSGVR3LRTNV2GEXSWQWSOYPHUOOHDJPYOUALO4M34TZGCHHBA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.620487996 (12h32m44.135590569s) [noise=true , address=104.36.82.61:4386 , piece_id=PNWQU2MPHMRVDXWMMWVNLWGLFQAMEJRLGJXFRCT4M3QYDEG5GSFQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.620594571 (12h32m44.135697142s) [noise=true , address=77.90.31.14:4683 , piece_id=4NNJC6OFHOZNQIKLWRGP7JOCMJAF5NHGTF24A6TID7JVKXNFZATQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.625884604 (12h32m44.140987167s) [noise=true , address=104.36.82.15:4267 , piece_id=DSRR5EHTM75MM7P4MG2YNPHWAE7WHALDCRE4L3YZKKH2BMKRTKAA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.626396996 (12h32m44.141499578s) [noise=true , address=104.36.82.58:4292 , piece_id=PFHGP3WHANRR4CSDD3VN5MBXQ65OV7UGJ5WDUP34AZYQZAMRLU7A, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.629464012 (12h32m44.144566574s) [noise=true , address=104.36.82.5:4844 , piece_id=GDXNIVT4KT5XFLFAR7B7NMHQDCE2OGGJQYHWID5ASAWO2IVNBN7Q, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.643428107 (12h32m44.158530675s) [noise=true , address=104.36.82.14:4248 , piece_id=7ZRBZNDCIDWWONITBOTR3OKLOWVCZ5VCANY6YAQQW6JYSUBUMJZA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.643719221 (12h32m44.15882179s) [noise=true , address=104.36.82.44:4873 , piece_id=GLETUFDVURZ3XPA2OHFTOA7NLKIQSPKF6HVFNPGBOOPU7ICLEOAA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.644290441 (12h32m44.159393003s) [noise=true , address=104.36.82.34:4575 , piece_id=MQE24KHKGXFJPHPKZH4NAI3G2FSOWA42JGN2WEXU47J6ALXEMFOQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.646798488 (12h32m44.161901065s) [noise=true , address=104.36.82.43:4856 , piece_id=PIKKNIOZICWM7EJM3NSRIJWPLBXIAP7WNT7RBREAHU6JRXZDCHYA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.656991689 (12h32m44.172094259s) [noise=true , address=104.36.82.56:4257 , piece_id=6CTH2UDV6ECR5GOBNLG36WD63IHMQTVYH3IUYH33RZYB4ZQ6LQ2A, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.835525769 (12h32m44.350628346s) [noise=true , address=79.127.201.215:4357 , piece_id=I2SQGM7NHIUHKNQG6RT3PH3W6P5OKJPP3MFJ65QC7CTN36DUYYJA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:10.167762322 (12h32m44.682864893s) [noise=true , address=64.49.13.154:4934 , piece_id=5T4YXQHU4DJEZCEAHQT3U4P636W5WDGSQCVCB73XQOCJFAV4O2MA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:10.167785966 (12h32m44.682888542s) [seg_pos={0 0}, upload=12956]: Segment reached optimal threshold of 14 pieces.
Aug 7 22:12:10.167971590 (12h32m44.683074167s) [noise=true , address=64.49.13.137:4493 , piece_id=432XMLABWE3DSWXU3B3BTREMG6CP7ILTNU4RJPA6JLHEJNX5POIA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12N91KV4t7yYJvKLFMaXHjByCZpsNgWFnYyZHJE7BqEYBGMLAi9): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168073316 (12h32m44.683175878s) [noise=true , address=104.36.82.18:4383 , piece_id=XVON7B5A6KIUDIZUJKNMNDQBQ5JNTCORJO7JL4NLPYOIGYU63TXQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Wtu7qHz3z5gYVPLJzaKKH3orBrtBUEVUeHRUaUe4xJGZv4u4z): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168146785 (12h32m44.683249347s) [noise=true , address=64.49.13.138:4490 , piece_id=RT5AQYMACT6OGCNBPVOJOLYGYQHHND6TVS3FEE52MXZWXZJLCCVA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Jp1vcm8dpppQHEfAtYmM29CDS3mqhojhuVrzkL3PsYHW2q8zA): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168258506 (12h32m44.683361075s) [noise=true , address=79.127.205.235:4670 , piece_id=3GWZPOLPZFZUAJKZ6MOXRTGGE7YDBMNP53WLJXEVOYW2ES5MLZPA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JgTfMzsd3rMqQwNim6BoZSiePoLW3gcUtraj84Kn4Szy7aNga): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168271822 (12h32m44.68337439s) [seg_pos={0 0}, upload=12956]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 22:12:10.168367951 (12h32m44.683470527s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 22:12:10.168372753 (12h32m44.683475329s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 22:12:11.355196489 (12h32m45.870299059s) [upload=12956]: Done uploading segments...
Aug 7 22:12:11.355214126 (12h32m45.870316692s) [upload=12956]: Upload finished. err: <nil>
Duration: 608.581441ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
27 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXX.XX.X..X.............................XXXXXX] duration:
30.000658359s
FAIL (2025-08-07 22:25:08 UTC)
operation: sync2.Copy uplink: encryption: context deadline exceeded
Duration: 30.000679237s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
24 out of 60 is failed (threshold=3) [XXXXXXXXXXX.X.................................XXXXXXXXXXX.X.] duration:
266.103104ms
FAIL (2025-08-07 22:25:07 UTC)
operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1527233 (12h35m13.476629807s) [upload=12995]: Uploading segments... Aug 7 22:14:38.961574411 (12h35m13.476676997s) [upload=12995]: Got next segment. Inline: true Aug 7 22:14:38.961580546 (12h35m13.47668312s) [upload=12995]: Waiting for error group managing segments... Aug 7 22:14:38.961644878 (12h35m13.476747457s) [upload=12995]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:14:38.961650553 (12h35m13.476753133s) [upload=12995]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:14:38.961655154 (12h35m13.476757734s) [upload=12995]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:15:38.926876179 (12h36m13.441978762s) [upload=12995]: Done uploading segments... Aug 7 22:15:38.926898754 (12h36m13.442001334s) [upload=12995]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:15:39.023006560 (12h36m13.538109126s) [upload=13009]: Starting upload Aug 7 22:15:39.023159911 (12h36m13.538262487s) [upload=13009]: Done starting upload Aug 7 22:15:39.023273936 (12h36m13.538376511s) [upload=13009]: Uploading segments... Aug 7 22:15:39.023358183 (12h36m13.538460767s) [upload=13009]: Got next segment. Inline: true Aug 7 22:15:39.023363552 (12h36m13.538466129s) [upload=13009]: Waiting for error group managing segments... Aug 7 22:15:39.023458469 (12h36m13.538561049s) [upload=13009]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:15:39.023464439 (12h36m13.538567019s) [upload=13009]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:15:39.023491067 (12h36m13.53859365s) [upload=13009]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:16:38.991189889 (12h37m13.506292477s) [upload=13009]: Done uploading segments... Aug 7 22:16:38.991214272 (12h37m13.506316848s) [upload=13009]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:16:39.116496595 (12h37m13.631599182s) [upload=13020]: Starting upload Aug 7 22:16:39.116578790 (12h37m13.631681366s) [upload=13020]: Done starting upload Aug 7 22:16:39.116734512 (12h37m13.631837083s) [upload=13020]: Uploading segments... Aug 7 22:16:39.116797388 (12h37m13.631899986s) [upload=13020]: Got next segment. Inline: true Aug 7 22:16:39.116803839 (12h37m13.631906421s) [upload=13020]: Waiting for error group managing segments... Aug 7 22:16:39.116855539 (12h37m13.631958104s) [upload=13020]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:16:39.116860739 (12h37m13.631963318s) [upload=13020]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:16:39.116864309 (12h37m13.631966875s) [upload=13020]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:17:39.087205419 (12h38m13.602307996s) [upload=13020]: Done uploading segments... Aug 7 22:17:39.087240451 (12h38m13.602343028s) [upload=13020]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:17:39.195279517 (12h38m13.710382082s) [upload=13033]: Starting upload Aug 7 22:17:39.195345128 (12h38m13.710447708s) [upload=13033]: Done starting upload Aug 7 22:17:39.195408125 (12h38m13.710510701s) [upload=13033]: Uploading segments... Aug 7 22:17:39.195557815 (12h38m13.710660402s) [upload=13033]: Got next segment. Inline: true Aug 7 22:17:39.195566565 (12h38m13.710669149s) [upload=13033]: Waiting for error group managing segments... Aug 7 22:17:39.195637820 (12h38m13.710740397s) [upload=13033]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:17:39.195643205 (12h38m13.710745782s) [upload=13033]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:39.195646438 (12h38m13.710749014s) [upload=13033]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:18:39.157680356 (12h39m13.672782934s) [upload=13033]: Done uploading segments... Aug 7 22:18:39.157703138 (12h39m13.672805715s) [upload=13033]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:18:39.252626432 (12h39m13.767729009s) [upload=13048]: Starting upload Aug 7 22:18:39.252783205 (12h39m13.767885845s) [upload=13048]: Done starting upload Aug 7 22:18:39.252852957 (12h39m13.767955535s) [upload=13048]: Uploading segments... Aug 7 22:18:39.252949510 (12h39m13.768052096s) [upload=13048]: Got next segment. Inline: true Aug 7 22:18:39.252955069 (12h39m13.768057637s) [upload=13048]: Waiting for error group managing segments... Aug 7 22:18:39.253031088 (12h39m13.76813365s) [upload=13048]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:18:39.253036915 (12h39m13.768139478s) [upload=13048]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:18:39.253040017 (12h39m13.768142595s) [upload=13048]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:19:39.223199412 (12h40m13.738302239s) [upload=13048]: Done uploading segments... Aug 7 22:19:39.223230125 (12h40m13.738332769s) [upload=13048]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:19:39.328604668 (12h40m13.843707247s) [upload=13065]: Starting upload Aug 7 22:19:39.328870365 (12h40m13.843972956s) [upload=13065]: Done starting upload Aug 7 22:19:39.329172418 (12h40m13.844274986s) [upload=13065]: Uploading segments... Aug 7 22:19:39.329244607 (12h40m13.844347193s) [upload=13065]: Got next segment. Inline: true Aug 7 22:19:39.329250690 (12h40m13.844353268s) [upload=13065]: Waiting for error group managing segments... Aug 7 22:19:39.329302352 (12h40m13.844404927s) [upload=13065]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:19:39.329309321 (12h40m13.844411902s) [upload=13065]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:19:39.329313964 (12h40m13.844416551s) [upload=13065]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:20:39.300343613 (12h41m13.81544618s) [upload=13065]: Done uploading segments... Aug 7 22:20:39.300363356 (12h41m13.815465922s) [upload=13065]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:45.259131358 (12h41m19.774233925s) [upload=13078]: Starting upload Aug 7 22:20:45.259192437 (12h41m19.774295013s) [upload=13078]: Done starting upload Aug 7 22:20:45.259251322 (12h41m19.774353897s) [upload=13078]: Uploading segments... Aug 7 22:20:45.259289829 (12h41m19.774392414s) [upload=13078]: Got next segment. Inline: true Aug 7 22:20:45.259294478 (12h41m19.774397054s) [upload=13078]: Waiting for error group managing segments... Aug 7 22:20:45.259332525 (12h41m19.774435101s) [upload=13078]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:20:45.259337069 (12h41m19.774439635s) [upload=13078]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:45.259340263 (12h41m19.77444284s) [upload=13078]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:21:45.229308083 (12h42m19.744410669s) [upload=13078]: Done uploading segments... Aug 7 22:21:45.229380638 (12h42m19.744483223s) [upload=13078]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:21:59.915154556 (12h42m34.430257139s) [upload=13097]: Starting upload Aug 7 22:21:59.915242531 (12h42m34.430345108s) [upload=13097]: Done starting upload Aug 7 22:21:59.915358515 (12h42m34.430461092s) [upload=13097]: Uploading segments... Aug 7 22:21:59.915398067 (12h42m34.430500653s) [upload=13097]: Got next segment. Inline: true Aug 7 22:21:59.915403151 (12h42m34.430505727s) [upload=13097]: Waiting for error group managing segments... Aug 7 22:21:59.915458296 (12h42m34.430560863s) [upload=13097]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:21:59.915463422 (12h42m34.430565998s) [upload=13097]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:21:59.915466628 (12h42m34.430569269s) [upload=13097]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:22:59.883702618 (12h43m34.398805207s) [upload=13097]: Done uploading segments... Aug 7 22:22:59.883744759 (12h43m34.398847337s) [upload=13097]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:00.217702003 (12h43m34.732804585s) [upload=13110]: Starting upload Aug 7 22:23:00.217805899 (12h43m34.732908483s) [upload=13110]: Done starting upload Aug 7 22:23:00.217896494 (12h43m34.732999072s) [upload=13110]: Uploading segments... Aug 7 22:23:00.217970831 (12h43m34.733073418s) [upload=13110]: Got next segment. Inline: true Aug 7 22:23:00.217979414 (12h43m34.733081991s) [upload=13110]: Waiting for error group managing segments... Aug 7 22:23:00.218043817 (12h43m34.733146394s) [upload=13110]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:23:00.218051159 (12h43m34.733153734s) [upload=13110]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:00.218054412 (12h43m34.733156988s) [upload=13110]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:23:00.464344185 (12h43m34.979446778s) [upload=13110]: Done uploading segments... Aug 7 22:23:00.464376970 (12h43m34.979479551s) [upload=13110]: Upload finished. err: <nil> Aug 7 22:24:07.143696312 (12h44m41.658798899s) [upload=13125]: Starting upload Aug 7 22:24:07.143811006 (12h44m41.658913589s) [upload=13125]: Done starting upload Aug 7 22:24:07.143914011 (12h44m41.659016577s) [upload=13125]: Uploading segments... Aug 7 22:24:07.143973667 (12h44m41.659076252s) [upload=13125]: Got next segment. Inline: true Aug 7 22:24:07.143979447 (12h44m41.659082034s) [upload=13125]: Waiting for error group managing segments... Aug 7 22:24:07.144045718 (12h44m41.659148297s) [upload=13125]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:24:07.144051783 (12h44m41.659154361s) [upload=13125]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:07.144057848 (12h44m41.659160429s) [upload=13125]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:25:07.110905816 (12h45m41.626008396s) [upload=13125]: Done uploading segments... Aug 7 22:25:07.110929433 (12h45m41.626032014s) [upload=13125]: Upload finished. err: metaclient: context deadline exceeded
Duration: 266.125309ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
26 out of 60 is failed (threshold=3) [XXXXXXXXXX.XX...X.............................XXXXXXXXXXX.XX] duration:
1m0.000848872s
FAIL (2025-08-07 22:25:01 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000897444s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
28 out of 60 is failed (threshold=3) [XXXXXX.XX.X..X..X..........................XXXXXXXXXXXXXXXXX] duration:
30.00063104s
FAIL (2025-08-07 22:24:56 UTC)
operation: Commit uplink: metaclient: context deadline exceeded
Duration: 30.000648711s
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
5 out of 60 is failed (threshold=3) [.XX................................................X..XX....] duration:
44.072225ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-07 22:24:56 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 44.093143ms
edge
gateway
vhost
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
21 out of 60 is failed (threshold=3) [.X....X.X.................................XXXXXXXXXXXXXXXXXX] duration:
30.001127562s
FAIL (2025-08-07 22:24:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001154488s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
24 out of 60 is failed (threshold=3) [XXXX....XX................................XXXXXXXXXXXXXXXXXX] duration:
30.000530638s
FAIL (2025-08-07 22:24:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.00054623s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
25 out of 60 is failed (threshold=3) [XXX.XX...XXX...............................XXXXXXXXXXXXXXXXX] duration:
30.00077026s
FAIL (2025-08-07 22:24:25 UTC)
operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 78262 (12h37m48.17058083s) [upload=13028]: Uploading segments... Aug 7 22:17:13.655548537 (12h37m48.17065113s) [upload=13028]: Got next segment. Inline: true Aug 7 22:17:13.655555256 (12h37m48.170657826s) [upload=13028]: Waiting for error group managing segments... Aug 7 22:17:13.655620901 (12h37m48.170723469s) [upload=13028]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:17:13.655627928 (12h37m48.170730508s) [upload=13028]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:13.655632202 (12h37m48.170734786s) [upload=13028]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:17:43.624980277 (12h38m18.140082854s) [upload=13028]: Done uploading segments... Aug 7 22:17:43.624997725 (12h38m18.140100301s) [upload=13028]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:17:43.747114121 (12h38m18.2622167s) [upload=13037]: Starting upload Aug 7 22:17:43.747186113 (12h38m18.262288692s) [upload=13037]: Done starting upload Aug 7 22:17:43.747260024 (12h38m18.262362591s) [upload=13037]: Uploading segments... Aug 7 22:17:43.747409410 (12h38m18.262511987s) [upload=13037]: Got next segment. Inline: true Aug 7 22:17:43.747414935 (12h38m18.262517512s) [upload=13037]: Waiting for error group managing segments... Aug 7 22:17:43.747457457 (12h38m18.262560036s) [upload=13037]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:17:43.747461997 (12h38m18.262564572s) [upload=13037]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:43.747465159 (12h38m18.262567736s) [upload=13037]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:18:13.715508579 (12h38m48.230611159s) [upload=13037]: Done uploading segments... Aug 7 22:18:13.715531957 (12h38m48.230634536s) [upload=13037]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:18:33.297944141 (12h39m7.813046721s) [upload=13046]: Starting upload Aug 7 22:18:33.298041745 (12h39m7.813144331s) [upload=13046]: Done starting upload Aug 7 22:18:33.298133963 (12h39m7.813236527s) [upload=13046]: Uploading segments... Aug 7 22:18:33.298195648 (12h39m7.813298236s) [upload=13046]: Got next segment. Inline: true Aug 7 22:18:33.298201348 (12h39m7.813303934s) [upload=13046]: Waiting for error group managing segments... Aug 7 22:18:33.298250060 (12h39m7.813352642s) [upload=13046]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:18:33.298255690 (12h39m7.813358273s) [upload=13046]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:18:33.298259809 (12h39m7.813362387s) [upload=13046]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:19:03.266409729 (12h39m37.781512306s) [upload=13046]: Done uploading segments... Aug 7 22:19:03.266458971 (12h39m37.781561549s) [upload=13046]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:19:30.020553557 (12h40m4.535656125s) [upload=13060]: Starting upload Aug 7 22:19:30.020630355 (12h40m4.535732931s) [upload=13060]: Done starting upload Aug 7 22:19:30.020870620 (12h40m4.535973198s) [upload=13060]: Uploading segments... Aug 7 22:19:30.020984540 (12h40m4.536087103s) [upload=13060]: Got next segment. Inline: true Aug 7 22:19:30.020990929 (12h40m4.536093523s) [upload=13060]: Waiting for error group managing segments... Aug 7 22:19:30.021057882 (12h40m4.536160461s) [upload=13060]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:19:30.021063707 (12h40m4.536166287s) [upload=13060]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:19:30.021067070 (12h40m4.536169633s) [upload=13060]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:19:59.990140275 (12h40m34.505242847s) [upload=13060]: Done uploading segments... Aug 7 22:19:59.990166176 (12h40m34.505268763s) [upload=13060]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:14.462825235 (12h40m48.977927811s) [upload=13071]: Starting upload Aug 7 22:20:14.462946370 (12h40m48.978048965s) [upload=13071]: Done starting upload Aug 7 22:20:14.463094242 (12h40m48.978196809s) [upload=13071]: Uploading segments... Aug 7 22:20:14.463162579 (12h40m48.978265164s) [upload=13071]: Got next segment. Inline: true Aug 7 22:20:14.463168150 (12h40m48.978270729s) [upload=13071]: Waiting for error group managing segments... Aug 7 22:20:14.463216881 (12h40m48.978319457s) [upload=13071]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:20:14.463226562 (12h40m48.978329138s) [upload=13071]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:14.463230088 (12h40m48.978332654s) [upload=13071]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:20:44.419309349 (12h41m18.934411917s) [upload=13071]: Done uploading segments... Aug 7 22:20:44.419327995 (12h41m18.934430565s) [upload=13071]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:59.026227228 (12h41m33.541329811s) [upload=13081]: Starting upload Aug 7 22:20:59.026335834 (12h41m33.541438465s) [upload=13081]: Done starting upload Aug 7 22:20:59.026413148 (12h41m33.541515725s) [upload=13081]: Uploading segments... Aug 7 22:20:59.026487957 (12h41m33.541590544s) [upload=13081]: Got next segment. Inline: true Aug 7 22:20:59.026494660 (12h41m33.54159724s) [upload=13081]: Waiting for error group managing segments... Aug 7 22:20:59.026548283 (12h41m33.541650867s) [upload=13081]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:20:59.026554014 (12h41m33.541656591s) [upload=13081]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:59.026557057 (12h41m33.541659621s) [upload=13081]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:21:28.995870666 (12h42m3.510973249s) [upload=13081]: Done uploading segments... Aug 7 22:21:28.995890051 (12h42m3.510992681s) [upload=13081]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:21:29.129769938 (12h42m3.644872515s) [upload=13090]: Starting upload Aug 7 22:21:29.129856429 (12h42m3.644959013s) [upload=13090]: Done starting upload Aug 7 22:21:29.129943698 (12h42m3.645046276s) [upload=13090]: Uploading segments... Aug 7 22:21:29.129985259 (12h42m3.645087845s) [upload=13090]: Got next segment. Inline: true Aug 7 22:21:29.129990476 (12h42m3.645093052s) [upload=13090]: Waiting for error group managing segments... Aug 7 22:21:29.130027566 (12h42m3.645130145s) [upload=13090]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:21:29.130031892 (12h42m3.645134469s) [upload=13090]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:21:29.130035507 (12h42m3.645138082s) [upload=13090]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:21:59.086555047 (12h42m33.601657631s) [upload=13090]: Done uploading segments... Aug 7 22:21:59.086578788 (12h42m33.601681377s) [upload=13090]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:22:21.787894978 (12h42m56.302997579s) [upload=13102]: Starting upload Aug 7 22:22:21.787975806 (12h42m56.303078386s) [upload=13102]: Done starting upload Aug 7 22:22:21.788049720 (12h42m56.303152297s) [upload=13102]: Uploading segments... Aug 7 22:22:21.788092210 (12h42m56.303194795s) [upload=13102]: Got next segment. Inline: true Aug 7 22:22:21.788097559 (12h42m56.303200126s) [upload=13102]: Waiting for error group managing segments... Aug 7 22:22:21.788141372 (12h42m56.303243949s) [upload=13102]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:22:21.788146123 (12h42m56.30324876s) [upload=13102]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:21.788149819 (12h42m56.303252395s) [upload=13102]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:22:51.758466888 (12h43m26.273569473s) [upload=13102]: Done uploading segments... Aug 7 22:22:51.758489416 (12h43m26.273591984s) [upload=13102]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:17.561525115 (12h43m52.076627681s) [upload=13117]: Starting upload Aug 7 22:23:17.561618947 (12h43m52.076721524s) [upload=13117]: Done starting upload Aug 7 22:23:17.561741793 (12h43m52.07684436s) [upload=13117]: Uploading segments... Aug 7 22:23:17.561843225 (12h43m52.076945802s) [upload=13117]: Got next segment. Inline: true Aug 7 22:23:17.561848606 (12h43m52.076951175s) [upload=13117]: Waiting for error group managing segments... Aug 7 22:23:17.561909318 (12h43m52.077011885s) [upload=13117]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:23:17.561914564 (12h43m52.077017202s) [upload=13117]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:17.561918167 (12h43m52.077020738s) [upload=13117]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:23:47.532068483 (12h44m22.047171052s) [upload=13117]: Done uploading segments... Aug 7 22:23:47.532095698 (12h44m22.047198268s) [upload=13117]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:55.295508312 (12h44m29.810610893s) [upload=13123]: Starting upload Aug 7 22:23:55.295607041 (12h44m29.81070962s) [upload=13123]: Done starting upload Aug 7 22:23:55.295735905 (12h44m29.810838474s) [upload=13123]: Uploading segments... Aug 7 22:23:55.295799978 (12h44m29.810902554s) [upload=13123]: Got next segment. Inline: true Aug 7 22:23:55.295805637 (12h44m29.810908214s) [upload=13123]: Waiting for error group managing segments... Aug 7 22:23:55.295849955 (12h44m29.810952532s) [upload=13123]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:23:55.295854812 (12h44m29.810957387s) [upload=13123]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:55.295858141 (12h44m29.810960717s) [upload=13123]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:24:25.257624495 (12h44m59.772727074s) [upload=13123]: Done uploading segments... Aug 7 22:24:25.257647063 (12h44m59.77274964s) [upload=13123]: Upload finished. err: metaclient: context deadline exceeded
Duration: 30.000808551s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
5 out of 60 is failed (threshold=3) [..XX................................................X..XX...] duration:
55.1437ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-07 22:24:25 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 55.168218ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..X.........................................X...............] duration:
1.258198024s
FAIL (2025-08-07 22:24:16 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 70b8ffb330ca946ca16cbba98295b25e9d73c23ceaf12561cb0410c721509eb0 != before 7a7c50f9088555b2e513a4e1d17c684f3e108f16da50d5cb1fb6d77de72f8388
======LOGS======
oad finished. err: <nil>
Aug 7 22:12:09.370677362 (12h32m43.885779928s) [upload=12956]: Starting upload
Aug 7 22:12:09.370743926 (12h32m43.885846503s) [upload=12956]: Done starting upload
Aug 7 22:12:09.371065555 (12h32m43.886168183s) [upload=12956]: Uploading segments...
Aug 7 22:12:09.371127350 (12h32m43.886229927s) [upload=12956]: Got next segment. Inline: false
Aug 7 22:12:09.371179942 (12h32m43.886282519s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 22:12:09.371184578 (12h32m43.886287156s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 22:12:09.514484447 (12h32m44.029587032s) [seg_pos={0 0}, upload=12956]: Begin upload segment...
Aug 7 22:12:09.514553324 (12h32m44.029655914s) [seg_pos={0 0}, upload=12956]: Done begin upload segment.
Aug 7 22:12:09.514604463 (12h32m44.029707051s) [noise=true , address=64.49.13.138:4490 , piece_id=RT5AQYMACT6OGCNBPVOJOLYGYQHHND6TVS3FEE52MXZWXZJLCCVA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.514656287 (12h32m44.029758882s) [noise=true , address=104.36.82.34:4575 , piece_id=MQE24KHKGXFJPHPKZH4NAI3G2FSOWA42JGN2WEXU47J6ALXEMFOQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.515221313 (12h32m44.030323899s) [noise=true , address=104.36.82.44:4873 , piece_id=GLETUFDVURZ3XPA2OHFTOA7NLKIQSPKF6HVFNPGBOOPU7ICLEOAA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.515435936 (12h32m44.030538517s) [noise=true , address=104.36.82.59:4326 , piece_id=SHCRSGVR3LRTNV2GEXSWQWSOYPHUOOHDJPYOUALO4M34TZGCHHBA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.515609669 (12h32m44.030712245s) [noise=true , address=79.127.201.215:4357 , piece_id=I2SQGM7NHIUHKNQG6RT3PH3W6P5OKJPP3MFJ65QC7CTN36DUYYJA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516008521 (12h32m44.03111111s) [noise=true , address=104.36.82.56:4257 , piece_id=6CTH2UDV6ECR5GOBNLG36WD63IHMQTVYH3IUYH33RZYB4ZQ6LQ2A, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516150437 (12h32m44.031253023s) [noise=true , address=45.91.116.11:4132 , piece_id=7DYDCKW5NLR22DRUCWIMELTSHM325ESELDERG26BTA2YLWQE2HHA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516344626 (12h32m44.031447208s) [noise=true , address=79.127.205.235:4670 , piece_id=3GWZPOLPZFZUAJKZ6MOXRTGGE7YDBMNP53WLJXEVOYW2ES5MLZPA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516841797 (12h32m44.031944374s) [noise=true , address=64.49.13.154:4934 , piece_id=5T4YXQHU4DJEZCEAHQT3U4P636W5WDGSQCVCB73XQOCJFAV4O2MA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517033011 (12h32m44.032135594s) [noise=true , address=104.36.82.14:4248 , piece_id=7ZRBZNDCIDWWONITBOTR3OKLOWVCZ5VCANY6YAQQW6JYSUBUMJZA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517240434 (12h32m44.032343025s) [noise=true , address=104.36.82.58:4292 , piece_id=PFHGP3WHANRR4CSDD3VN5MBXQ65OV7UGJ5WDUP34AZYQZAMRLU7A, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517628325 (12h32m44.032730907s) [noise=true , address=64.49.13.137:4493 , piece_id=432XMLABWE3DSWXU3B3BTREMG6CP7ILTNU4RJPA6JLHEJNX5POIA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517733543 (12h32m44.032836126s) [noise=true , address=104.36.82.61:4386 , piece_id=PNWQU2MPHMRVDXWMMWVNLWGLFQAMEJRLGJXFRCT4M3QYDEG5GSFQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.518003213 (12h32m44.033105792s) [noise=true , address=104.36.82.5:4844 , piece_id=GDXNIVT4KT5XFLFAR7B7NMHQDCE2OGGJQYHWID5ASAWO2IVNBN7Q, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.518348003 (12h32m44.03345058s) [noise=true , address=77.90.31.14:4683 , piece_id=4NNJC6OFHOZNQIKLWRGP7JOCMJAF5NHGTF24A6TID7JVKXNFZATQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.518755006 (12h32m44.033857585s) [noise=true , address=104.36.82.15:4267 , piece_id=DSRR5EHTM75MM7P4MG2YNPHWAE7WHALDCRE4L3YZKKH2BMKRTKAA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.522094892 (12h32m44.037197472s) [noise=true , address=104.36.82.18:4383 , piece_id=XVON7B5A6KIUDIZUJKNMNDQBQ5JNTCORJO7JL4NLPYOIGYU63TXQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.523039320 (12h32m44.038141897s) [noise=true , address=104.36.82.43:4856 , piece_id=PIKKNIOZICWM7EJM3NSRIJWPLBXIAP7WNT7RBREAHU6JRXZDCHYA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.590300484 (12h32m44.105403049s) [upload=12956]: Next returned nil segment
Aug 7 22:12:09.590317010 (12h32m44.105419591s) [upload=12956]: Waiting for error group managing segments...
Aug 7 22:12:09.604186438 (12h32m44.119289008s) [noise=true , address=45.91.116.11:4132 , piece_id=7DYDCKW5NLR22DRUCWIMELTSHM325ESELDERG26BTA2YLWQE2HHA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.606286681 (12h32m44.121389287s) [noise=true , address=104.36.82.59:4326 , piece_id=SHCRSGVR3LRTNV2GEXSWQWSOYPHUOOHDJPYOUALO4M34TZGCHHBA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.620487996 (12h32m44.135590569s) [noise=true , address=104.36.82.61:4386 , piece_id=PNWQU2MPHMRVDXWMMWVNLWGLFQAMEJRLGJXFRCT4M3QYDEG5GSFQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.620594571 (12h32m44.135697142s) [noise=true , address=77.90.31.14:4683 , piece_id=4NNJC6OFHOZNQIKLWRGP7JOCMJAF5NHGTF24A6TID7JVKXNFZATQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.625884604 (12h32m44.140987167s) [noise=true , address=104.36.82.15:4267 , piece_id=DSRR5EHTM75MM7P4MG2YNPHWAE7WHALDCRE4L3YZKKH2BMKRTKAA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.626396996 (12h32m44.141499578s) [noise=true , address=104.36.82.58:4292 , piece_id=PFHGP3WHANRR4CSDD3VN5MBXQ65OV7UGJ5WDUP34AZYQZAMRLU7A, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.629464012 (12h32m44.144566574s) [noise=true , address=104.36.82.5:4844 , piece_id=GDXNIVT4KT5XFLFAR7B7NMHQDCE2OGGJQYHWID5ASAWO2IVNBN7Q, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.643428107 (12h32m44.158530675s) [noise=true , address=104.36.82.14:4248 , piece_id=7ZRBZNDCIDWWONITBOTR3OKLOWVCZ5VCANY6YAQQW6JYSUBUMJZA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.643719221 (12h32m44.15882179s) [noise=true , address=104.36.82.44:4873 , piece_id=GLETUFDVURZ3XPA2OHFTOA7NLKIQSPKF6HVFNPGBOOPU7ICLEOAA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.644290441 (12h32m44.159393003s) [noise=true , address=104.36.82.34:4575 , piece_id=MQE24KHKGXFJPHPKZH4NAI3G2FSOWA42JGN2WEXU47J6ALXEMFOQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.646798488 (12h32m44.161901065s) [noise=true , address=104.36.82.43:4856 , piece_id=PIKKNIOZICWM7EJM3NSRIJWPLBXIAP7WNT7RBREAHU6JRXZDCHYA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.656991689 (12h32m44.172094259s) [noise=true , address=104.36.82.56:4257 , piece_id=6CTH2UDV6ECR5GOBNLG36WD63IHMQTVYH3IUYH33RZYB4ZQ6LQ2A, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.835525769 (12h32m44.350628346s) [noise=true , address=79.127.201.215:4357 , piece_id=I2SQGM7NHIUHKNQG6RT3PH3W6P5OKJPP3MFJ65QC7CTN36DUYYJA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:10.167762322 (12h32m44.682864893s) [noise=true , address=64.49.13.154:4934 , piece_id=5T4YXQHU4DJEZCEAHQT3U4P636W5WDGSQCVCB73XQOCJFAV4O2MA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:10.167785966 (12h32m44.682888542s) [seg_pos={0 0}, upload=12956]: Segment reached optimal threshold of 14 pieces.
Aug 7 22:12:10.167971590 (12h32m44.683074167s) [noise=true , address=64.49.13.137:4493 , piece_id=432XMLABWE3DSWXU3B3BTREMG6CP7ILTNU4RJPA6JLHEJNX5POIA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12N91KV4t7yYJvKLFMaXHjByCZpsNgWFnYyZHJE7BqEYBGMLAi9): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168073316 (12h32m44.683175878s) [noise=true , address=104.36.82.18:4383 , piece_id=XVON7B5A6KIUDIZUJKNMNDQBQ5JNTCORJO7JL4NLPYOIGYU63TXQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Wtu7qHz3z5gYVPLJzaKKH3orBrtBUEVUeHRUaUe4xJGZv4u4z): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168146785 (12h32m44.683249347s) [noise=true , address=64.49.13.138:4490 , piece_id=RT5AQYMACT6OGCNBPVOJOLYGYQHHND6TVS3FEE52MXZWXZJLCCVA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Jp1vcm8dpppQHEfAtYmM29CDS3mqhojhuVrzkL3PsYHW2q8zA): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168258506 (12h32m44.683361075s) [noise=true , address=79.127.205.235:4670 , piece_id=3GWZPOLPZFZUAJKZ6MOXRTGGE7YDBMNP53WLJXEVOYW2ES5MLZPA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JgTfMzsd3rMqQwNim6BoZSiePoLW3gcUtraj84Kn4Szy7aNga): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168271822 (12h32m44.68337439s) [seg_pos={0 0}, upload=12956]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 22:12:10.168367951 (12h32m44.683470527s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 22:12:10.168372753 (12h32m44.683475329s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 22:12:11.355196489 (12h32m45.870299059s) [upload=12956]: Done uploading segments...
Aug 7 22:12:11.355214126 (12h32m45.870316692s) [upload=12956]: Upload finished. err: <nil>
Duration: 1.258227169s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
24 out of 60 is failed (threshold=3) [XXXXX....XX................................XXXXXXXXXXXXXXXXX] duration:
30.001314856s
FAIL (2025-08-07 22:24:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.0013474s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
26 out of 60 is failed (threshold=3) [XXXXXXXXXXX.XX...X.............................XXXXXXXXXXX.X] duration:
1m0.000617197s
FAIL (2025-08-07 22:24:01 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000662091s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
21 out of 60 is failed (threshold=3) [X.X....X.X.................................XXXXXXXXXXXXXXXXX] duration:
30.001364631s
FAIL (2025-08-07 22:23:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001398242s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
27 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXX.XX.X..X.............................XXXXX] duration:
30.000697167s
FAIL (2025-08-07 22:23:55 UTC)
operation: sync2.Copy uplink: encryption: context deadline exceeded
Duration: 30.000732532s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
25 out of 60 is failed (threshold=3) [XXXX.XX...XXX...............................XXXXXXXXXXXXXXXX] duration:
30.000549375s
FAIL (2025-08-07 22:23:47 UTC)
operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 488 (12h36m54.243237051s) [upload=13016]: Uploading segments... Aug 7 22:16:19.728182154 (12h36m54.243284736s) [upload=13016]: Got next segment. Inline: true Aug 7 22:16:19.728186380 (12h36m54.243288955s) [upload=13016]: Waiting for error group managing segments... Aug 7 22:16:19.728223320 (12h36m54.243325899s) [upload=13016]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:16:19.728227150 (12h36m54.243329727s) [upload=13016]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:16:19.728229527 (12h36m54.243332103s) [upload=13016]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:16:49.691229995 (12h37m24.206332575s) [upload=13016]: Done uploading segments... Aug 7 22:16:49.691252411 (12h37m24.206354988s) [upload=13016]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:17:13.655336229 (12h37m48.170438797s) [upload=13028]: Starting upload Aug 7 22:17:13.655409916 (12h37m48.170512492s) [upload=13028]: Done starting upload Aug 7 22:17:13.655478262 (12h37m48.17058083s) [upload=13028]: Uploading segments... Aug 7 22:17:13.655548537 (12h37m48.17065113s) [upload=13028]: Got next segment. Inline: true Aug 7 22:17:13.655555256 (12h37m48.170657826s) [upload=13028]: Waiting for error group managing segments... Aug 7 22:17:13.655620901 (12h37m48.170723469s) [upload=13028]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:17:13.655627928 (12h37m48.170730508s) [upload=13028]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:13.655632202 (12h37m48.170734786s) [upload=13028]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:17:43.624980277 (12h38m18.140082854s) [upload=13028]: Done uploading segments... Aug 7 22:17:43.624997725 (12h38m18.140100301s) [upload=13028]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:17:43.747114121 (12h38m18.2622167s) [upload=13037]: Starting upload Aug 7 22:17:43.747186113 (12h38m18.262288692s) [upload=13037]: Done starting upload Aug 7 22:17:43.747260024 (12h38m18.262362591s) [upload=13037]: Uploading segments... Aug 7 22:17:43.747409410 (12h38m18.262511987s) [upload=13037]: Got next segment. Inline: true Aug 7 22:17:43.747414935 (12h38m18.262517512s) [upload=13037]: Waiting for error group managing segments... Aug 7 22:17:43.747457457 (12h38m18.262560036s) [upload=13037]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:17:43.747461997 (12h38m18.262564572s) [upload=13037]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:43.747465159 (12h38m18.262567736s) [upload=13037]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:18:13.715508579 (12h38m48.230611159s) [upload=13037]: Done uploading segments... Aug 7 22:18:13.715531957 (12h38m48.230634536s) [upload=13037]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:18:33.297944141 (12h39m7.813046721s) [upload=13046]: Starting upload Aug 7 22:18:33.298041745 (12h39m7.813144331s) [upload=13046]: Done starting upload Aug 7 22:18:33.298133963 (12h39m7.813236527s) [upload=13046]: Uploading segments... Aug 7 22:18:33.298195648 (12h39m7.813298236s) [upload=13046]: Got next segment. Inline: true Aug 7 22:18:33.298201348 (12h39m7.813303934s) [upload=13046]: Waiting for error group managing segments... Aug 7 22:18:33.298250060 (12h39m7.813352642s) [upload=13046]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:18:33.298255690 (12h39m7.813358273s) [upload=13046]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:18:33.298259809 (12h39m7.813362387s) [upload=13046]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:19:03.266409729 (12h39m37.781512306s) [upload=13046]: Done uploading segments... Aug 7 22:19:03.266458971 (12h39m37.781561549s) [upload=13046]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:19:30.020553557 (12h40m4.535656125s) [upload=13060]: Starting upload Aug 7 22:19:30.020630355 (12h40m4.535732931s) [upload=13060]: Done starting upload Aug 7 22:19:30.020870620 (12h40m4.535973198s) [upload=13060]: Uploading segments... Aug 7 22:19:30.020984540 (12h40m4.536087103s) [upload=13060]: Got next segment. Inline: true Aug 7 22:19:30.020990929 (12h40m4.536093523s) [upload=13060]: Waiting for error group managing segments... Aug 7 22:19:30.021057882 (12h40m4.536160461s) [upload=13060]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:19:30.021063707 (12h40m4.536166287s) [upload=13060]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:19:30.021067070 (12h40m4.536169633s) [upload=13060]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:19:59.990140275 (12h40m34.505242847s) [upload=13060]: Done uploading segments... Aug 7 22:19:59.990166176 (12h40m34.505268763s) [upload=13060]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:14.462825235 (12h40m48.977927811s) [upload=13071]: Starting upload Aug 7 22:20:14.462946370 (12h40m48.978048965s) [upload=13071]: Done starting upload Aug 7 22:20:14.463094242 (12h40m48.978196809s) [upload=13071]: Uploading segments... Aug 7 22:20:14.463162579 (12h40m48.978265164s) [upload=13071]: Got next segment. Inline: true Aug 7 22:20:14.463168150 (12h40m48.978270729s) [upload=13071]: Waiting for error group managing segments... Aug 7 22:20:14.463216881 (12h40m48.978319457s) [upload=13071]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:20:14.463226562 (12h40m48.978329138s) [upload=13071]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:14.463230088 (12h40m48.978332654s) [upload=13071]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:20:44.419309349 (12h41m18.934411917s) [upload=13071]: Done uploading segments... Aug 7 22:20:44.419327995 (12h41m18.934430565s) [upload=13071]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:59.026227228 (12h41m33.541329811s) [upload=13081]: Starting upload Aug 7 22:20:59.026335834 (12h41m33.541438465s) [upload=13081]: Done starting upload Aug 7 22:20:59.026413148 (12h41m33.541515725s) [upload=13081]: Uploading segments... Aug 7 22:20:59.026487957 (12h41m33.541590544s) [upload=13081]: Got next segment. Inline: true Aug 7 22:20:59.026494660 (12h41m33.54159724s) [upload=13081]: Waiting for error group managing segments... Aug 7 22:20:59.026548283 (12h41m33.541650867s) [upload=13081]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:20:59.026554014 (12h41m33.541656591s) [upload=13081]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:59.026557057 (12h41m33.541659621s) [upload=13081]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:21:28.995870666 (12h42m3.510973249s) [upload=13081]: Done uploading segments... Aug 7 22:21:28.995890051 (12h42m3.510992681s) [upload=13081]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:21:29.129769938 (12h42m3.644872515s) [upload=13090]: Starting upload Aug 7 22:21:29.129856429 (12h42m3.644959013s) [upload=13090]: Done starting upload Aug 7 22:21:29.129943698 (12h42m3.645046276s) [upload=13090]: Uploading segments... Aug 7 22:21:29.129985259 (12h42m3.645087845s) [upload=13090]: Got next segment. Inline: true Aug 7 22:21:29.129990476 (12h42m3.645093052s) [upload=13090]: Waiting for error group managing segments... Aug 7 22:21:29.130027566 (12h42m3.645130145s) [upload=13090]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:21:29.130031892 (12h42m3.645134469s) [upload=13090]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:21:29.130035507 (12h42m3.645138082s) [upload=13090]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:21:59.086555047 (12h42m33.601657631s) [upload=13090]: Done uploading segments... Aug 7 22:21:59.086578788 (12h42m33.601681377s) [upload=13090]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:22:21.787894978 (12h42m56.302997579s) [upload=13102]: Starting upload Aug 7 22:22:21.787975806 (12h42m56.303078386s) [upload=13102]: Done starting upload Aug 7 22:22:21.788049720 (12h42m56.303152297s) [upload=13102]: Uploading segments... Aug 7 22:22:21.788092210 (12h42m56.303194795s) [upload=13102]: Got next segment. Inline: true Aug 7 22:22:21.788097559 (12h42m56.303200126s) [upload=13102]: Waiting for error group managing segments... Aug 7 22:22:21.788141372 (12h42m56.303243949s) [upload=13102]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:22:21.788146123 (12h42m56.30324876s) [upload=13102]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:21.788149819 (12h42m56.303252395s) [upload=13102]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:22:51.758466888 (12h43m26.273569473s) [upload=13102]: Done uploading segments... Aug 7 22:22:51.758489416 (12h43m26.273591984s) [upload=13102]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:17.561525115 (12h43m52.076627681s) [upload=13117]: Starting upload Aug 7 22:23:17.561618947 (12h43m52.076721524s) [upload=13117]: Done starting upload Aug 7 22:23:17.561741793 (12h43m52.07684436s) [upload=13117]: Uploading segments... Aug 7 22:23:17.561843225 (12h43m52.076945802s) [upload=13117]: Got next segment. Inline: true Aug 7 22:23:17.561848606 (12h43m52.076951175s) [upload=13117]: Waiting for error group managing segments... Aug 7 22:23:17.561909318 (12h43m52.077011885s) [upload=13117]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:23:17.561914564 (12h43m52.077017202s) [upload=13117]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:17.561918167 (12h43m52.077020738s) [upload=13117]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:23:47.532068483 (12h44m22.047171052s) [upload=13117]: Done uploading segments... Aug 7 22:23:47.532095698 (12h44m22.047198268s) [upload=13117]: Upload finished. err: metaclient: context deadline exceeded
Duration: 30.000572706s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
29 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXX.....X..........................XXXXXXXXXX] duration:
1m8.608794747s
FAIL (2025-08-07 22:23:45 UTC)
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
726798532 (12h43m19.241901115s) [seg_pos={0 0}, upload=13106]: Begin upload segment...
Aug 7 22:22:44.726884334 (12h43m19.24198692s) [noise=true , address=104.36.82.45:4919 , piece_id=6IGKH6GAIJSS55YT33KD7TM73DYF5K23QG4RBVEAEYM5PZFIHUOA, seg_pos={0 0}, upload=13106]: Uploading piece...
Aug 7 22:22:44.727108471 (12h43m19.242211051s) [seg_pos={0 0}, upload=13106]: Done begin upload segment.
Aug 7 22:22:44.727132404 (12h43m19.242234989s) [noise=true , address=104.36.82.47:4537 , piece_id=LPSV545AEXGMS3AEH6YG6OQBSXCAQPSIORWHLSY3FOWGJQEHVOOA, seg_pos={0 0}, upload=13106]: Uploading piece...
Aug 7 22:22:44.727869785 (12h43m19.24297237s) [noise=true , address=104.36.82.36:4632 , piece_id=F7DEYSZKTSFBUNPC32JK63255E7CB3G7GCUUH3AJIJ4R4EAE4WMA, seg_pos={0 0}, upload=13106]: Uploading piece...
Aug 7 22:22:44.728059514 (12h43m19.243162099s) [noise=true , address=104.36.82.35:4607 , piece_id=KBGWKPKFTGDOLZBIYR54UUTXFKFZFRM5QVBDMHQW47TVCBBYCAUA, seg_pos={0 0}, upload=13106]: Uploading piece...
Aug 7 22:22:44.728224725 (12h43m19.243327329s) [noise=true , address=79.127.205.233:4636 , piece_id=UIMIGGCRLQYFTIOVYIMWUOFRETS5663DV7RMKK74PZMOAZWS24LQ, seg_pos={0 0}, upload=13106]: Uploading piece...
Aug 7 22:22:44.729090322 (12h43m19.244192909s) [noise=true , address=104.36.82.37:4684 , piece_id=CUBHFRITSAUZS4N2KQJUS3MGYXPUAFXBQAMTTJYGB4DSYUNXFFPA, seg_pos={0 0}, upload=13106]: Uploading piece...
Aug 7 22:22:44.729125365 (12h43m19.244227945s) [noise=true , address=45.91.116.5:4301 , piece_id=ZAMXRRVNX2LA6LS4XKDQKSK7QMYRDXMSBDAOPM6QZFTCKSBL3PYA, seg_pos={0 0}, upload=13106]: Uploading piece...
Aug 7 22:22:44.730094526 (12h43m19.245197113s) [noise=true , address=104.36.82.9:4021 , piece_id=LBI2E2SMQADXDFT5SOH5772I3HYBRUS6UGUHQX6SDNJEZERKTVYQ, seg_pos={0 0}, upload=13106]: Uploading piece...
Aug 7 22:22:44.730151744 (12h43m19.245254324s) [noise=true , address=104.36.82.16:4311 , piece_id=SGEE2Q6JOYMHAJJMFK2A3EY63PEZ7CJCCIHGQUNIDSPHHBHG5YFQ, seg_pos={0 0}, upload=13106]: Uploading piece...
Aug 7 22:22:44.730912134 (12h43m19.246014744s) [noise=true , address=104.36.82.43:4836 , piece_id=TQIP77IGDME2IBNJWHRDNA34467PRBFJTWRHGAZUFYR6OC5KWVAA, seg_pos={0 0}, upload=13106]: Uploading piece...
Aug 7 22:22:44.731166404 (12h43m19.246268991s) [noise=true , address=64.49.13.155:4918 , piece_id=H7JROCYDWIHDVAMA55G7JNMPVRP7FTXJRQ4MRNXXKPTUAG76FB6A, seg_pos={0 0}, upload=13106]: Uploading piece...
Aug 7 22:22:44.731624652 (12h43m19.246727239s) [noise=true , address=104.36.82.56:4241 , piece_id=2XKSB5JZS7R272H2CXC7PUWECF3AR6WICRKZQZON7H6LCP6N6JHQ, seg_pos={0 0}, upload=13106]: Uploading piece...
Aug 7 22:22:44.733456024 (12h43m19.24855861s) [noise=true , address=64.49.13.26:4953 , piece_id=7OPDDZFSIXNDKKAADV65AMPDDVDPTURWNU6AEWZV444IDJU3S4HA, seg_pos={0 0}, upload=13106]: Uploading piece...
Aug 7 22:22:44.733906772 (12h43m19.249009359s) [noise=true , address=77.90.31.21:4897 , piece_id=H2OLXBUR4JL65GJCCNNRK6MU7PDH6MNSR7ELH4LAQQHMBDXSE6FA, seg_pos={0 0}, upload=13106]: Uploading piece...
Aug 7 22:22:44.738487566 (12h43m19.253590152s) [noise=true , address=104.36.82.62:4436 , piece_id=6S3AQNJRSZXS6Z5RHUNFUBVUP4CKEUSAS2UDJVUHAZQXDFKQX7WQ, seg_pos={0 0}, upload=13106]: Uploading piece...
Aug 7 22:22:44.739259686 (12h43m19.254362267s) [noise=true , address=104.36.82.9:4017 , piece_id=6KTMFJ3RUYN7J25RE62FLIOSJ2WDA623GY7KZMNCAJI3MG5MFROQ, seg_pos={0 0}, upload=13106]: Uploading piece...
Aug 7 22:22:44.740050099 (12h43m19.255152685s) [noise=true , address=104.36.82.61:4405 , piece_id=PTQA2EGCMTHQWFSIMXUQYIRQS7JWGRILUUYSMNPDWMHIHBWJTQLQ, seg_pos={0 0}, upload=13106]: Uploading piece...
Aug 7 22:22:44.742328280 (12h43m19.257430857s) [noise=true , address=104.36.82.10:4110 , piece_id=KNS7DWUMJS2NCQCCY2X2PCHFE6ZK5QEDHKAEM4X6CXXDCNP25FDQ, seg_pos={0 0}, upload=13106]: Uploading piece...
Aug 7 22:22:44.813600791 (12h43m19.328703359s) [upload=13106]: Next returned nil segment
Aug 7 22:22:44.813617280 (12h43m19.328719846s) [upload=13106]: Waiting for error group managing segments...
Aug 7 22:22:44.830075283 (12h43m19.34517786s) [noise=true , address=45.91.116.5:4301 , piece_id=ZAMXRRVNX2LA6LS4XKDQKSK7QMYRDXMSBDAOPM6QZFTCKSBL3PYA, seg_pos={0 0}, upload=13106]: Done uploading piece. err: <nil>
Aug 7 22:22:44.830213489 (12h43m19.345316058s) [noise=true , address=104.36.82.16:4311 , piece_id=SGEE2Q6JOYMHAJJMFK2A3EY63PEZ7CJCCIHGQUNIDSPHHBHG5YFQ, seg_pos={0 0}, upload=13106]: Done uploading piece. err: <nil>
Aug 7 22:22:44.835139283 (12h43m19.350241853s) [noise=true , address=104.36.82.35:4607 , piece_id=KBGWKPKFTGDOLZBIYR54UUTXFKFZFRM5QVBDMHQW47TVCBBYCAUA, seg_pos={0 0}, upload=13106]: Done uploading piece. err: <nil>
Aug 7 22:22:44.836868151 (12h43m19.35197073s) [noise=true , address=104.36.82.43:4836 , piece_id=TQIP77IGDME2IBNJWHRDNA34467PRBFJTWRHGAZUFYR6OC5KWVAA, seg_pos={0 0}, upload=13106]: Done uploading piece. err: <nil>
Aug 7 22:22:44.840331887 (12h43m19.355434464s) [noise=true , address=104.36.82.45:4919 , piece_id=6IGKH6GAIJSS55YT33KD7TM73DYF5K23QG4RBVEAEYM5PZFIHUOA, seg_pos={0 0}, upload=13106]: Done uploading piece. err: <nil>
Aug 7 22:22:44.844141126 (12h43m19.359243695s) [noise=true , address=104.36.82.47:4537 , piece_id=LPSV545AEXGMS3AEH6YG6OQBSXCAQPSIORWHLSY3FOWGJQEHVOOA, seg_pos={0 0}, upload=13106]: Done uploading piece. err: <nil>
Aug 7 22:22:44.848344813 (12h43m19.363447396s) [noise=true , address=77.90.31.21:4897 , piece_id=H2OLXBUR4JL65GJCCNNRK6MU7PDH6MNSR7ELH4LAQQHMBDXSE6FA, seg_pos={0 0}, upload=13106]: Done uploading piece. err: <nil>
Aug 7 22:22:44.871391183 (12h43m19.386493746s) [noise=true , address=104.36.82.9:4017 , piece_id=6KTMFJ3RUYN7J25RE62FLIOSJ2WDA623GY7KZMNCAJI3MG5MFROQ, seg_pos={0 0}, upload=13106]: Done uploading piece. err: <nil>
Aug 7 22:22:44.881104660 (12h43m19.396207238s) [noise=true , address=104.36.82.56:4241 , piece_id=2XKSB5JZS7R272H2CXC7PUWECF3AR6WICRKZQZON7H6LCP6N6JHQ, seg_pos={0 0}, upload=13106]: Done uploading piece. err: <nil>
Aug 7 22:22:44.947378281 (12h43m19.462480893s) [noise=true , address=104.36.82.9:4021 , piece_id=LBI2E2SMQADXDFT5SOH5772I3HYBRUS6UGUHQX6SDNJEZERKTVYQ, seg_pos={0 0}, upload=13106]: Done uploading piece. err: <nil>
Aug 7 22:22:45.080443216 (12h43m19.595545786s) [noise=true , address=104.36.82.36:4632 , piece_id=F7DEYSZKTSFBUNPC32JK63255E7CB3G7GCUUH3AJIJ4R4EAE4WMA, seg_pos={0 0}, upload=13106]: Done uploading piece. err: <nil>
Aug 7 22:22:45.119250997 (12h43m19.634353562s) [noise=true , address=104.36.82.10:4110 , piece_id=KNS7DWUMJS2NCQCCY2X2PCHFE6ZK5QEDHKAEM4X6CXXDCNP25FDQ, seg_pos={0 0}, upload=13106]: Done uploading piece. err: <nil>
Aug 7 22:22:45.342696636 (12h43m19.857799214s) [noise=true , address=79.127.205.233:4636 , piece_id=UIMIGGCRLQYFTIOVYIMWUOFRETS5663DV7RMKK74PZMOAZWS24LQ, seg_pos={0 0}, upload=13106]: Done uploading piece. err: <nil>
Aug 7 22:22:45.381606983 (12h43m19.896709551s) [noise=true , address=64.49.13.155:4918 , piece_id=H7JROCYDWIHDVAMA55G7JNMPVRP7FTXJRQ4MRNXXKPTUAG76FB6A, seg_pos={0 0}, upload=13106]: Done uploading piece. err: <nil>
Aug 7 22:22:45.381643649 (12h43m19.896746239s) [seg_pos={0 0}, upload=13106]: Segment reached optimal threshold of 14 pieces.
Aug 7 22:22:45.381920874 (12h43m19.897023454s) [noise=true , address=64.49.13.26:4953 , piece_id=7OPDDZFSIXNDKKAADV65AMPDDVDPTURWNU6AEWZV444IDJU3S4HA, seg_pos={0 0}, upload=13106]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: expected piece hash; context canceled; EOF
Aug 7 22:22:45.381967530 (12h43m19.897070112s) [noise=true , address=104.36.82.61:4405 , piece_id=PTQA2EGCMTHQWFSIMXUQYIRQS7JWGRILUUYSMNPDWMHIHBWJTQLQ, seg_pos={0 0}, upload=13106]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XPFXQaLxDzhZjcaACurTejzdA8S4szrXVJjc5pgJm9UMreNiy): protocol: write tcp 159.223.143.39:55372->104.36.82.61:4405: use of closed network connection; write tcp 159.223.143.39:55372->104.36.82.61:4405: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:55372->104.36.82.61:4405: use of closed network connection
Aug 7 22:22:45.382079774 (12h43m19.897182341s) [noise=true , address=104.36.82.62:4436 , piece_id=6S3AQNJRSZXS6Z5RHUNFUBVUP4CKEUSAS2UDJVUHAZQXDFKQX7WQ, seg_pos={0 0}, upload=13106]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QrrGrMHYAjyYiTdTXDUy2uH1j2p1qtUVJaDtais2GowJqBPPu): protocol: read tcp 159.223.143.39:41224->104.36.82.62:4436: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 7 22:22:45.382185759 (12h43m19.897288338s) [noise=true , address=104.36.82.37:4684 , piece_id=CUBHFRITSAUZS4N2KQJUS3MGYXPUAFXBQAMTTJYGB4DSYUNXFFPA, seg_pos={0 0}, upload=13106]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123rwT3AmQR5nrvjoo6fgKBBJaKVWJHkJ6FtLnoNYs2AxakoDZC): protocol: write tcp 159.223.143.39:42930->104.36.82.37:4684: use of closed network connection; write tcp 159.223.143.39:42930->104.36.82.37:4684: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42930->104.36.82.37:4684: use of closed network connection
Aug 7 22:22:45.382216876 (12h43m19.897319453s) [seg_pos={0 0}, upload=13106]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 22:22:45.382315173 (12h43m19.897417749s) [upload=13106]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 22:22:45.382320719 (12h43m19.897423286s) [upload=13106]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 22:23:45.712335424 (12h44m20.227437993s) [upload=13106]: Done uploading segments...
Aug 7 22:23:45.712370252 (12h44m20.227472829s) [upload=13106]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1m8.608814959s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
28 out of 60 is failed (threshold=3) [XXXXXXX.XX.X..X..X..........................XXXXXXXXXXXXXXXX] duration:
30.001047886s
FAIL (2025-08-07 22:23:44 UTC)
operation: Commit uplink: metaclient: context deadline exceeded
Duration: 30.001071876s
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
6 out of 60 is failed (threshold=3) [X..XX................................................X..XX..] duration:
75.234009ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-07 22:23:28 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 75.266535ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...X.........................................X..............] duration:
1.793171479s
FAIL (2025-08-07 22:23:17 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 70b8ffb330ca946ca16cbba98295b25e9d73c23ceaf12561cb0410c721509eb0 != before 7a7c50f9088555b2e513a4e1d17c684f3e108f16da50d5cb1fb6d77de72f8388
======LOGS======
oad finished. err: <nil>
Aug 7 22:12:09.370677362 (12h32m43.885779928s) [upload=12956]: Starting upload
Aug 7 22:12:09.370743926 (12h32m43.885846503s) [upload=12956]: Done starting upload
Aug 7 22:12:09.371065555 (12h32m43.886168183s) [upload=12956]: Uploading segments...
Aug 7 22:12:09.371127350 (12h32m43.886229927s) [upload=12956]: Got next segment. Inline: false
Aug 7 22:12:09.371179942 (12h32m43.886282519s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 22:12:09.371184578 (12h32m43.886287156s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 22:12:09.514484447 (12h32m44.029587032s) [seg_pos={0 0}, upload=12956]: Begin upload segment...
Aug 7 22:12:09.514553324 (12h32m44.029655914s) [seg_pos={0 0}, upload=12956]: Done begin upload segment.
Aug 7 22:12:09.514604463 (12h32m44.029707051s) [noise=true , address=64.49.13.138:4490 , piece_id=RT5AQYMACT6OGCNBPVOJOLYGYQHHND6TVS3FEE52MXZWXZJLCCVA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.514656287 (12h32m44.029758882s) [noise=true , address=104.36.82.34:4575 , piece_id=MQE24KHKGXFJPHPKZH4NAI3G2FSOWA42JGN2WEXU47J6ALXEMFOQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.515221313 (12h32m44.030323899s) [noise=true , address=104.36.82.44:4873 , piece_id=GLETUFDVURZ3XPA2OHFTOA7NLKIQSPKF6HVFNPGBOOPU7ICLEOAA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.515435936 (12h32m44.030538517s) [noise=true , address=104.36.82.59:4326 , piece_id=SHCRSGVR3LRTNV2GEXSWQWSOYPHUOOHDJPYOUALO4M34TZGCHHBA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.515609669 (12h32m44.030712245s) [noise=true , address=79.127.201.215:4357 , piece_id=I2SQGM7NHIUHKNQG6RT3PH3W6P5OKJPP3MFJ65QC7CTN36DUYYJA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516008521 (12h32m44.03111111s) [noise=true , address=104.36.82.56:4257 , piece_id=6CTH2UDV6ECR5GOBNLG36WD63IHMQTVYH3IUYH33RZYB4ZQ6LQ2A, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516150437 (12h32m44.031253023s) [noise=true , address=45.91.116.11:4132 , piece_id=7DYDCKW5NLR22DRUCWIMELTSHM325ESELDERG26BTA2YLWQE2HHA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516344626 (12h32m44.031447208s) [noise=true , address=79.127.205.235:4670 , piece_id=3GWZPOLPZFZUAJKZ6MOXRTGGE7YDBMNP53WLJXEVOYW2ES5MLZPA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516841797 (12h32m44.031944374s) [noise=true , address=64.49.13.154:4934 , piece_id=5T4YXQHU4DJEZCEAHQT3U4P636W5WDGSQCVCB73XQOCJFAV4O2MA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517033011 (12h32m44.032135594s) [noise=true , address=104.36.82.14:4248 , piece_id=7ZRBZNDCIDWWONITBOTR3OKLOWVCZ5VCANY6YAQQW6JYSUBUMJZA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517240434 (12h32m44.032343025s) [noise=true , address=104.36.82.58:4292 , piece_id=PFHGP3WHANRR4CSDD3VN5MBXQ65OV7UGJ5WDUP34AZYQZAMRLU7A, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517628325 (12h32m44.032730907s) [noise=true , address=64.49.13.137:4493 , piece_id=432XMLABWE3DSWXU3B3BTREMG6CP7ILTNU4RJPA6JLHEJNX5POIA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517733543 (12h32m44.032836126s) [noise=true , address=104.36.82.61:4386 , piece_id=PNWQU2MPHMRVDXWMMWVNLWGLFQAMEJRLGJXFRCT4M3QYDEG5GSFQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.518003213 (12h32m44.033105792s) [noise=true , address=104.36.82.5:4844 , piece_id=GDXNIVT4KT5XFLFAR7B7NMHQDCE2OGGJQYHWID5ASAWO2IVNBN7Q, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.518348003 (12h32m44.03345058s) [noise=true , address=77.90.31.14:4683 , piece_id=4NNJC6OFHOZNQIKLWRGP7JOCMJAF5NHGTF24A6TID7JVKXNFZATQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.518755006 (12h32m44.033857585s) [noise=true , address=104.36.82.15:4267 , piece_id=DSRR5EHTM75MM7P4MG2YNPHWAE7WHALDCRE4L3YZKKH2BMKRTKAA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.522094892 (12h32m44.037197472s) [noise=true , address=104.36.82.18:4383 , piece_id=XVON7B5A6KIUDIZUJKNMNDQBQ5JNTCORJO7JL4NLPYOIGYU63TXQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.523039320 (12h32m44.038141897s) [noise=true , address=104.36.82.43:4856 , piece_id=PIKKNIOZICWM7EJM3NSRIJWPLBXIAP7WNT7RBREAHU6JRXZDCHYA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.590300484 (12h32m44.105403049s) [upload=12956]: Next returned nil segment
Aug 7 22:12:09.590317010 (12h32m44.105419591s) [upload=12956]: Waiting for error group managing segments...
Aug 7 22:12:09.604186438 (12h32m44.119289008s) [noise=true , address=45.91.116.11:4132 , piece_id=7DYDCKW5NLR22DRUCWIMELTSHM325ESELDERG26BTA2YLWQE2HHA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.606286681 (12h32m44.121389287s) [noise=true , address=104.36.82.59:4326 , piece_id=SHCRSGVR3LRTNV2GEXSWQWSOYPHUOOHDJPYOUALO4M34TZGCHHBA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.620487996 (12h32m44.135590569s) [noise=true , address=104.36.82.61:4386 , piece_id=PNWQU2MPHMRVDXWMMWVNLWGLFQAMEJRLGJXFRCT4M3QYDEG5GSFQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.620594571 (12h32m44.135697142s) [noise=true , address=77.90.31.14:4683 , piece_id=4NNJC6OFHOZNQIKLWRGP7JOCMJAF5NHGTF24A6TID7JVKXNFZATQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.625884604 (12h32m44.140987167s) [noise=true , address=104.36.82.15:4267 , piece_id=DSRR5EHTM75MM7P4MG2YNPHWAE7WHALDCRE4L3YZKKH2BMKRTKAA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.626396996 (12h32m44.141499578s) [noise=true , address=104.36.82.58:4292 , piece_id=PFHGP3WHANRR4CSDD3VN5MBXQ65OV7UGJ5WDUP34AZYQZAMRLU7A, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.629464012 (12h32m44.144566574s) [noise=true , address=104.36.82.5:4844 , piece_id=GDXNIVT4KT5XFLFAR7B7NMHQDCE2OGGJQYHWID5ASAWO2IVNBN7Q, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.643428107 (12h32m44.158530675s) [noise=true , address=104.36.82.14:4248 , piece_id=7ZRBZNDCIDWWONITBOTR3OKLOWVCZ5VCANY6YAQQW6JYSUBUMJZA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.643719221 (12h32m44.15882179s) [noise=true , address=104.36.82.44:4873 , piece_id=GLETUFDVURZ3XPA2OHFTOA7NLKIQSPKF6HVFNPGBOOPU7ICLEOAA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.644290441 (12h32m44.159393003s) [noise=true , address=104.36.82.34:4575 , piece_id=MQE24KHKGXFJPHPKZH4NAI3G2FSOWA42JGN2WEXU47J6ALXEMFOQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.646798488 (12h32m44.161901065s) [noise=true , address=104.36.82.43:4856 , piece_id=PIKKNIOZICWM7EJM3NSRIJWPLBXIAP7WNT7RBREAHU6JRXZDCHYA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.656991689 (12h32m44.172094259s) [noise=true , address=104.36.82.56:4257 , piece_id=6CTH2UDV6ECR5GOBNLG36WD63IHMQTVYH3IUYH33RZYB4ZQ6LQ2A, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.835525769 (12h32m44.350628346s) [noise=true , address=79.127.201.215:4357 , piece_id=I2SQGM7NHIUHKNQG6RT3PH3W6P5OKJPP3MFJ65QC7CTN36DUYYJA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:10.167762322 (12h32m44.682864893s) [noise=true , address=64.49.13.154:4934 , piece_id=5T4YXQHU4DJEZCEAHQT3U4P636W5WDGSQCVCB73XQOCJFAV4O2MA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:10.167785966 (12h32m44.682888542s) [seg_pos={0 0}, upload=12956]: Segment reached optimal threshold of 14 pieces.
Aug 7 22:12:10.167971590 (12h32m44.683074167s) [noise=true , address=64.49.13.137:4493 , piece_id=432XMLABWE3DSWXU3B3BTREMG6CP7ILTNU4RJPA6JLHEJNX5POIA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12N91KV4t7yYJvKLFMaXHjByCZpsNgWFnYyZHJE7BqEYBGMLAi9): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168073316 (12h32m44.683175878s) [noise=true , address=104.36.82.18:4383 , piece_id=XVON7B5A6KIUDIZUJKNMNDQBQ5JNTCORJO7JL4NLPYOIGYU63TXQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Wtu7qHz3z5gYVPLJzaKKH3orBrtBUEVUeHRUaUe4xJGZv4u4z): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168146785 (12h32m44.683249347s) [noise=true , address=64.49.13.138:4490 , piece_id=RT5AQYMACT6OGCNBPVOJOLYGYQHHND6TVS3FEE52MXZWXZJLCCVA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Jp1vcm8dpppQHEfAtYmM29CDS3mqhojhuVrzkL3PsYHW2q8zA): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168258506 (12h32m44.683361075s) [noise=true , address=79.127.205.235:4670 , piece_id=3GWZPOLPZFZUAJKZ6MOXRTGGE7YDBMNP53WLJXEVOYW2ES5MLZPA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JgTfMzsd3rMqQwNim6BoZSiePoLW3gcUtraj84Kn4Szy7aNga): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168271822 (12h32m44.68337439s) [seg_pos={0 0}, upload=12956]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 22:12:10.168367951 (12h32m44.683470527s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 22:12:10.168372753 (12h32m44.683475329s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 22:12:11.355196489 (12h32m45.870299059s) [upload=12956]: Done uploading segments...
Aug 7 22:12:11.355214126 (12h32m45.870316692s) [upload=12956]: Upload finished. err: <nil>
Duration: 1.793215919s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
21 out of 60 is failed (threshold=3) [XX.X....X.X.................................XXXXXXXXXXXXXXXX] duration:
30.000292589s
FAIL (2025-08-07 22:23:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000311037s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
25 out of 60 is failed (threshold=3) [XXXXXXXXXXXXX.X.................................XXXXXXXXXXX.] duration:
767.612646ms
FAIL (2025-08-07 22:23:00 UTC)
operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 3m8.347189539s) [upload=12965]: Uploading segments... Aug 7 22:12:33.832135510 (12h33m8.347238093s) [upload=12965]: Got next segment. Inline: true Aug 7 22:12:33.832141249 (12h33m8.347243826s) [upload=12965]: Waiting for error group managing segments... Aug 7 22:12:33.832194093 (12h33m8.347296674s) [upload=12965]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:12:33.832199847 (12h33m8.347302426s) [upload=12965]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:12:33.832205088 (12h33m8.347307666s) [upload=12965]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:13:33.802839016 (12h34m8.317941593s) [upload=12965]: Done uploading segments... Aug 7 22:13:33.802913313 (12h34m8.31801589s) [upload=12965]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:13:33.913275619 (12h34m8.428378249s) [upload=12977]: Starting upload Aug 7 22:13:33.913398357 (12h34m8.428500941s) [upload=12977]: Done starting upload Aug 7 22:13:33.913496779 (12h34m8.428599348s) [upload=12977]: Uploading segments... Aug 7 22:13:33.913568818 (12h34m8.428671424s) [upload=12977]: Got next segment. Inline: true Aug 7 22:13:33.913576485 (12h34m8.428679072s) [upload=12977]: Waiting for error group managing segments... Aug 7 22:13:33.913646668 (12h34m8.428749234s) [upload=12977]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:13:33.913652247 (12h34m8.428754834s) [upload=12977]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:13:33.913655814 (12h34m8.428758404s) [upload=12977]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:14:33.874545239 (12h35m8.389647805s) [upload=12977]: Done uploading segments... Aug 7 22:14:33.874575517 (12h35m8.389678082s) [upload=12977]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:14:38.961368058 (12h35m13.476470627s) [upload=12995]: Starting upload Aug 7 22:14:38.961455294 (12h35m13.476557856s) [upload=12995]: Done starting upload Aug 7 22:14:38.961527233 (12h35m13.476629807s) [upload=12995]: Uploading segments... Aug 7 22:14:38.961574411 (12h35m13.476676997s) [upload=12995]: Got next segment. Inline: true Aug 7 22:14:38.961580546 (12h35m13.47668312s) [upload=12995]: Waiting for error group managing segments... Aug 7 22:14:38.961644878 (12h35m13.476747457s) [upload=12995]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:14:38.961650553 (12h35m13.476753133s) [upload=12995]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:14:38.961655154 (12h35m13.476757734s) [upload=12995]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:15:38.926876179 (12h36m13.441978762s) [upload=12995]: Done uploading segments... Aug 7 22:15:38.926898754 (12h36m13.442001334s) [upload=12995]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:15:39.023006560 (12h36m13.538109126s) [upload=13009]: Starting upload Aug 7 22:15:39.023159911 (12h36m13.538262487s) [upload=13009]: Done starting upload Aug 7 22:15:39.023273936 (12h36m13.538376511s) [upload=13009]: Uploading segments... Aug 7 22:15:39.023358183 (12h36m13.538460767s) [upload=13009]: Got next segment. Inline: true Aug 7 22:15:39.023363552 (12h36m13.538466129s) [upload=13009]: Waiting for error group managing segments... Aug 7 22:15:39.023458469 (12h36m13.538561049s) [upload=13009]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:15:39.023464439 (12h36m13.538567019s) [upload=13009]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:15:39.023491067 (12h36m13.53859365s) [upload=13009]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:16:38.991189889 (12h37m13.506292477s) [upload=13009]: Done uploading segments... Aug 7 22:16:38.991214272 (12h37m13.506316848s) [upload=13009]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:16:39.116496595 (12h37m13.631599182s) [upload=13020]: Starting upload Aug 7 22:16:39.116578790 (12h37m13.631681366s) [upload=13020]: Done starting upload Aug 7 22:16:39.116734512 (12h37m13.631837083s) [upload=13020]: Uploading segments... Aug 7 22:16:39.116797388 (12h37m13.631899986s) [upload=13020]: Got next segment. Inline: true Aug 7 22:16:39.116803839 (12h37m13.631906421s) [upload=13020]: Waiting for error group managing segments... Aug 7 22:16:39.116855539 (12h37m13.631958104s) [upload=13020]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:16:39.116860739 (12h37m13.631963318s) [upload=13020]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:16:39.116864309 (12h37m13.631966875s) [upload=13020]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:17:39.087205419 (12h38m13.602307996s) [upload=13020]: Done uploading segments... Aug 7 22:17:39.087240451 (12h38m13.602343028s) [upload=13020]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:17:39.195279517 (12h38m13.710382082s) [upload=13033]: Starting upload Aug 7 22:17:39.195345128 (12h38m13.710447708s) [upload=13033]: Done starting upload Aug 7 22:17:39.195408125 (12h38m13.710510701s) [upload=13033]: Uploading segments... Aug 7 22:17:39.195557815 (12h38m13.710660402s) [upload=13033]: Got next segment. Inline: true Aug 7 22:17:39.195566565 (12h38m13.710669149s) [upload=13033]: Waiting for error group managing segments... Aug 7 22:17:39.195637820 (12h38m13.710740397s) [upload=13033]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:17:39.195643205 (12h38m13.710745782s) [upload=13033]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:39.195646438 (12h38m13.710749014s) [upload=13033]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:18:39.157680356 (12h39m13.672782934s) [upload=13033]: Done uploading segments... Aug 7 22:18:39.157703138 (12h39m13.672805715s) [upload=13033]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:18:39.252626432 (12h39m13.767729009s) [upload=13048]: Starting upload Aug 7 22:18:39.252783205 (12h39m13.767885845s) [upload=13048]: Done starting upload Aug 7 22:18:39.252852957 (12h39m13.767955535s) [upload=13048]: Uploading segments... Aug 7 22:18:39.252949510 (12h39m13.768052096s) [upload=13048]: Got next segment. Inline: true Aug 7 22:18:39.252955069 (12h39m13.768057637s) [upload=13048]: Waiting for error group managing segments... Aug 7 22:18:39.253031088 (12h39m13.76813365s) [upload=13048]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:18:39.253036915 (12h39m13.768139478s) [upload=13048]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:18:39.253040017 (12h39m13.768142595s) [upload=13048]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:19:39.223199412 (12h40m13.738302239s) [upload=13048]: Done uploading segments... Aug 7 22:19:39.223230125 (12h40m13.738332769s) [upload=13048]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:19:39.328604668 (12h40m13.843707247s) [upload=13065]: Starting upload Aug 7 22:19:39.328870365 (12h40m13.843972956s) [upload=13065]: Done starting upload Aug 7 22:19:39.329172418 (12h40m13.844274986s) [upload=13065]: Uploading segments... Aug 7 22:19:39.329244607 (12h40m13.844347193s) [upload=13065]: Got next segment. Inline: true Aug 7 22:19:39.329250690 (12h40m13.844353268s) [upload=13065]: Waiting for error group managing segments... Aug 7 22:19:39.329302352 (12h40m13.844404927s) [upload=13065]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:19:39.329309321 (12h40m13.844411902s) [upload=13065]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:19:39.329313964 (12h40m13.844416551s) [upload=13065]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:20:39.300343613 (12h41m13.81544618s) [upload=13065]: Done uploading segments... Aug 7 22:20:39.300363356 (12h41m13.815465922s) [upload=13065]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:45.259131358 (12h41m19.774233925s) [upload=13078]: Starting upload Aug 7 22:20:45.259192437 (12h41m19.774295013s) [upload=13078]: Done starting upload Aug 7 22:20:45.259251322 (12h41m19.774353897s) [upload=13078]: Uploading segments... Aug 7 22:20:45.259289829 (12h41m19.774392414s) [upload=13078]: Got next segment. Inline: true Aug 7 22:20:45.259294478 (12h41m19.774397054s) [upload=13078]: Waiting for error group managing segments... Aug 7 22:20:45.259332525 (12h41m19.774435101s) [upload=13078]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:20:45.259337069 (12h41m19.774439635s) [upload=13078]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:45.259340263 (12h41m19.77444284s) [upload=13078]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:21:45.229308083 (12h42m19.744410669s) [upload=13078]: Done uploading segments... Aug 7 22:21:45.229380638 (12h42m19.744483223s) [upload=13078]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:21:59.915154556 (12h42m34.430257139s) [upload=13097]: Starting upload Aug 7 22:21:59.915242531 (12h42m34.430345108s) [upload=13097]: Done starting upload Aug 7 22:21:59.915358515 (12h42m34.430461092s) [upload=13097]: Uploading segments... Aug 7 22:21:59.915398067 (12h42m34.430500653s) [upload=13097]: Got next segment. Inline: true Aug 7 22:21:59.915403151 (12h42m34.430505727s) [upload=13097]: Waiting for error group managing segments... Aug 7 22:21:59.915458296 (12h42m34.430560863s) [upload=13097]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:21:59.915463422 (12h42m34.430565998s) [upload=13097]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:21:59.915466628 (12h42m34.430569269s) [upload=13097]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:22:59.883702618 (12h43m34.398805207s) [upload=13097]: Done uploading segments... Aug 7 22:22:59.883744759 (12h43m34.398847337s) [upload=13097]: Upload finished. err: metaclient: context deadline exceeded
Duration: 767.635389ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
26 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXX.X.................................XXXXXXXXXXX] duration:
1m0.001133851s
FAIL (2025-08-07 22:22:59 UTC)
operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 3m8.347189539s) [upload=12965]: Uploading segments... Aug 7 22:12:33.832135510 (12h33m8.347238093s) [upload=12965]: Got next segment. Inline: true Aug 7 22:12:33.832141249 (12h33m8.347243826s) [upload=12965]: Waiting for error group managing segments... Aug 7 22:12:33.832194093 (12h33m8.347296674s) [upload=12965]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:12:33.832199847 (12h33m8.347302426s) [upload=12965]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:12:33.832205088 (12h33m8.347307666s) [upload=12965]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:13:33.802839016 (12h34m8.317941593s) [upload=12965]: Done uploading segments... Aug 7 22:13:33.802913313 (12h34m8.31801589s) [upload=12965]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:13:33.913275619 (12h34m8.428378249s) [upload=12977]: Starting upload Aug 7 22:13:33.913398357 (12h34m8.428500941s) [upload=12977]: Done starting upload Aug 7 22:13:33.913496779 (12h34m8.428599348s) [upload=12977]: Uploading segments... Aug 7 22:13:33.913568818 (12h34m8.428671424s) [upload=12977]: Got next segment. Inline: true Aug 7 22:13:33.913576485 (12h34m8.428679072s) [upload=12977]: Waiting for error group managing segments... Aug 7 22:13:33.913646668 (12h34m8.428749234s) [upload=12977]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:13:33.913652247 (12h34m8.428754834s) [upload=12977]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:13:33.913655814 (12h34m8.428758404s) [upload=12977]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:14:33.874545239 (12h35m8.389647805s) [upload=12977]: Done uploading segments... Aug 7 22:14:33.874575517 (12h35m8.389678082s) [upload=12977]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:14:38.961368058 (12h35m13.476470627s) [upload=12995]: Starting upload Aug 7 22:14:38.961455294 (12h35m13.476557856s) [upload=12995]: Done starting upload Aug 7 22:14:38.961527233 (12h35m13.476629807s) [upload=12995]: Uploading segments... Aug 7 22:14:38.961574411 (12h35m13.476676997s) [upload=12995]: Got next segment. Inline: true Aug 7 22:14:38.961580546 (12h35m13.47668312s) [upload=12995]: Waiting for error group managing segments... Aug 7 22:14:38.961644878 (12h35m13.476747457s) [upload=12995]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:14:38.961650553 (12h35m13.476753133s) [upload=12995]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:14:38.961655154 (12h35m13.476757734s) [upload=12995]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:15:38.926876179 (12h36m13.441978762s) [upload=12995]: Done uploading segments... Aug 7 22:15:38.926898754 (12h36m13.442001334s) [upload=12995]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:15:39.023006560 (12h36m13.538109126s) [upload=13009]: Starting upload Aug 7 22:15:39.023159911 (12h36m13.538262487s) [upload=13009]: Done starting upload Aug 7 22:15:39.023273936 (12h36m13.538376511s) [upload=13009]: Uploading segments... Aug 7 22:15:39.023358183 (12h36m13.538460767s) [upload=13009]: Got next segment. Inline: true Aug 7 22:15:39.023363552 (12h36m13.538466129s) [upload=13009]: Waiting for error group managing segments... Aug 7 22:15:39.023458469 (12h36m13.538561049s) [upload=13009]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:15:39.023464439 (12h36m13.538567019s) [upload=13009]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:15:39.023491067 (12h36m13.53859365s) [upload=13009]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:16:38.991189889 (12h37m13.506292477s) [upload=13009]: Done uploading segments... Aug 7 22:16:38.991214272 (12h37m13.506316848s) [upload=13009]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:16:39.116496595 (12h37m13.631599182s) [upload=13020]: Starting upload Aug 7 22:16:39.116578790 (12h37m13.631681366s) [upload=13020]: Done starting upload Aug 7 22:16:39.116734512 (12h37m13.631837083s) [upload=13020]: Uploading segments... Aug 7 22:16:39.116797388 (12h37m13.631899986s) [upload=13020]: Got next segment. Inline: true Aug 7 22:16:39.116803839 (12h37m13.631906421s) [upload=13020]: Waiting for error group managing segments... Aug 7 22:16:39.116855539 (12h37m13.631958104s) [upload=13020]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:16:39.116860739 (12h37m13.631963318s) [upload=13020]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:16:39.116864309 (12h37m13.631966875s) [upload=13020]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:17:39.087205419 (12h38m13.602307996s) [upload=13020]: Done uploading segments... Aug 7 22:17:39.087240451 (12h38m13.602343028s) [upload=13020]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:17:39.195279517 (12h38m13.710382082s) [upload=13033]: Starting upload Aug 7 22:17:39.195345128 (12h38m13.710447708s) [upload=13033]: Done starting upload Aug 7 22:17:39.195408125 (12h38m13.710510701s) [upload=13033]: Uploading segments... Aug 7 22:17:39.195557815 (12h38m13.710660402s) [upload=13033]: Got next segment. Inline: true Aug 7 22:17:39.195566565 (12h38m13.710669149s) [upload=13033]: Waiting for error group managing segments... Aug 7 22:17:39.195637820 (12h38m13.710740397s) [upload=13033]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:17:39.195643205 (12h38m13.710745782s) [upload=13033]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:39.195646438 (12h38m13.710749014s) [upload=13033]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:18:39.157680356 (12h39m13.672782934s) [upload=13033]: Done uploading segments... Aug 7 22:18:39.157703138 (12h39m13.672805715s) [upload=13033]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:18:39.252626432 (12h39m13.767729009s) [upload=13048]: Starting upload Aug 7 22:18:39.252783205 (12h39m13.767885845s) [upload=13048]: Done starting upload Aug 7 22:18:39.252852957 (12h39m13.767955535s) [upload=13048]: Uploading segments... Aug 7 22:18:39.252949510 (12h39m13.768052096s) [upload=13048]: Got next segment. Inline: true Aug 7 22:18:39.252955069 (12h39m13.768057637s) [upload=13048]: Waiting for error group managing segments... Aug 7 22:18:39.253031088 (12h39m13.76813365s) [upload=13048]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:18:39.253036915 (12h39m13.768139478s) [upload=13048]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:18:39.253040017 (12h39m13.768142595s) [upload=13048]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:19:39.223199412 (12h40m13.738302239s) [upload=13048]: Done uploading segments... Aug 7 22:19:39.223230125 (12h40m13.738332769s) [upload=13048]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:19:39.328604668 (12h40m13.843707247s) [upload=13065]: Starting upload Aug 7 22:19:39.328870365 (12h40m13.843972956s) [upload=13065]: Done starting upload Aug 7 22:19:39.329172418 (12h40m13.844274986s) [upload=13065]: Uploading segments... Aug 7 22:19:39.329244607 (12h40m13.844347193s) [upload=13065]: Got next segment. Inline: true Aug 7 22:19:39.329250690 (12h40m13.844353268s) [upload=13065]: Waiting for error group managing segments... Aug 7 22:19:39.329302352 (12h40m13.844404927s) [upload=13065]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:19:39.329309321 (12h40m13.844411902s) [upload=13065]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:19:39.329313964 (12h40m13.844416551s) [upload=13065]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:20:39.300343613 (12h41m13.81544618s) [upload=13065]: Done uploading segments... Aug 7 22:20:39.300363356 (12h41m13.815465922s) [upload=13065]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:45.259131358 (12h41m19.774233925s) [upload=13078]: Starting upload Aug 7 22:20:45.259192437 (12h41m19.774295013s) [upload=13078]: Done starting upload Aug 7 22:20:45.259251322 (12h41m19.774353897s) [upload=13078]: Uploading segments... Aug 7 22:20:45.259289829 (12h41m19.774392414s) [upload=13078]: Got next segment. Inline: true Aug 7 22:20:45.259294478 (12h41m19.774397054s) [upload=13078]: Waiting for error group managing segments... Aug 7 22:20:45.259332525 (12h41m19.774435101s) [upload=13078]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:20:45.259337069 (12h41m19.774439635s) [upload=13078]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:45.259340263 (12h41m19.77444284s) [upload=13078]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:21:45.229308083 (12h42m19.744410669s) [upload=13078]: Done uploading segments... Aug 7 22:21:45.229380638 (12h42m19.744483223s) [upload=13078]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:21:59.915154556 (12h42m34.430257139s) [upload=13097]: Starting upload Aug 7 22:21:59.915242531 (12h42m34.430345108s) [upload=13097]: Done starting upload Aug 7 22:21:59.915358515 (12h42m34.430461092s) [upload=13097]: Uploading segments... Aug 7 22:21:59.915398067 (12h42m34.430500653s) [upload=13097]: Got next segment. Inline: true Aug 7 22:21:59.915403151 (12h42m34.430505727s) [upload=13097]: Waiting for error group managing segments... Aug 7 22:21:59.915458296 (12h42m34.430560863s) [upload=13097]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:21:59.915463422 (12h42m34.430565998s) [upload=13097]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:21:59.915466628 (12h42m34.430569269s) [upload=13097]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:22:59.883702618 (12h43m34.398805207s) [upload=13097]: Done uploading segments... Aug 7 22:22:59.883744759 (12h43m34.398847337s) [upload=13097]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1m0.001159446s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
24 out of 60 is failed (threshold=3) [XXXXXX....XX................................XXXXXXXXXXXXXXXX] duration:
30.000791181s
FAIL (2025-08-07 22:22:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000815291s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [....X.........................................X.............] duration:
1.064142118s
FAIL (2025-08-07 22:22:53 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 70b8ffb330ca946ca16cbba98295b25e9d73c23ceaf12561cb0410c721509eb0 != before 7a7c50f9088555b2e513a4e1d17c684f3e108f16da50d5cb1fb6d77de72f8388
======LOGS======
oad finished. err: <nil>
Aug 7 22:12:09.370677362 (12h32m43.885779928s) [upload=12956]: Starting upload
Aug 7 22:12:09.370743926 (12h32m43.885846503s) [upload=12956]: Done starting upload
Aug 7 22:12:09.371065555 (12h32m43.886168183s) [upload=12956]: Uploading segments...
Aug 7 22:12:09.371127350 (12h32m43.886229927s) [upload=12956]: Got next segment. Inline: false
Aug 7 22:12:09.371179942 (12h32m43.886282519s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 22:12:09.371184578 (12h32m43.886287156s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 22:12:09.514484447 (12h32m44.029587032s) [seg_pos={0 0}, upload=12956]: Begin upload segment...
Aug 7 22:12:09.514553324 (12h32m44.029655914s) [seg_pos={0 0}, upload=12956]: Done begin upload segment.
Aug 7 22:12:09.514604463 (12h32m44.029707051s) [noise=true , address=64.49.13.138:4490 , piece_id=RT5AQYMACT6OGCNBPVOJOLYGYQHHND6TVS3FEE52MXZWXZJLCCVA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.514656287 (12h32m44.029758882s) [noise=true , address=104.36.82.34:4575 , piece_id=MQE24KHKGXFJPHPKZH4NAI3G2FSOWA42JGN2WEXU47J6ALXEMFOQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.515221313 (12h32m44.030323899s) [noise=true , address=104.36.82.44:4873 , piece_id=GLETUFDVURZ3XPA2OHFTOA7NLKIQSPKF6HVFNPGBOOPU7ICLEOAA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.515435936 (12h32m44.030538517s) [noise=true , address=104.36.82.59:4326 , piece_id=SHCRSGVR3LRTNV2GEXSWQWSOYPHUOOHDJPYOUALO4M34TZGCHHBA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.515609669 (12h32m44.030712245s) [noise=true , address=79.127.201.215:4357 , piece_id=I2SQGM7NHIUHKNQG6RT3PH3W6P5OKJPP3MFJ65QC7CTN36DUYYJA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516008521 (12h32m44.03111111s) [noise=true , address=104.36.82.56:4257 , piece_id=6CTH2UDV6ECR5GOBNLG36WD63IHMQTVYH3IUYH33RZYB4ZQ6LQ2A, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516150437 (12h32m44.031253023s) [noise=true , address=45.91.116.11:4132 , piece_id=7DYDCKW5NLR22DRUCWIMELTSHM325ESELDERG26BTA2YLWQE2HHA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516344626 (12h32m44.031447208s) [noise=true , address=79.127.205.235:4670 , piece_id=3GWZPOLPZFZUAJKZ6MOXRTGGE7YDBMNP53WLJXEVOYW2ES5MLZPA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.516841797 (12h32m44.031944374s) [noise=true , address=64.49.13.154:4934 , piece_id=5T4YXQHU4DJEZCEAHQT3U4P636W5WDGSQCVCB73XQOCJFAV4O2MA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517033011 (12h32m44.032135594s) [noise=true , address=104.36.82.14:4248 , piece_id=7ZRBZNDCIDWWONITBOTR3OKLOWVCZ5VCANY6YAQQW6JYSUBUMJZA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517240434 (12h32m44.032343025s) [noise=true , address=104.36.82.58:4292 , piece_id=PFHGP3WHANRR4CSDD3VN5MBXQ65OV7UGJ5WDUP34AZYQZAMRLU7A, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517628325 (12h32m44.032730907s) [noise=true , address=64.49.13.137:4493 , piece_id=432XMLABWE3DSWXU3B3BTREMG6CP7ILTNU4RJPA6JLHEJNX5POIA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.517733543 (12h32m44.032836126s) [noise=true , address=104.36.82.61:4386 , piece_id=PNWQU2MPHMRVDXWMMWVNLWGLFQAMEJRLGJXFRCT4M3QYDEG5GSFQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.518003213 (12h32m44.033105792s) [noise=true , address=104.36.82.5:4844 , piece_id=GDXNIVT4KT5XFLFAR7B7NMHQDCE2OGGJQYHWID5ASAWO2IVNBN7Q, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.518348003 (12h32m44.03345058s) [noise=true , address=77.90.31.14:4683 , piece_id=4NNJC6OFHOZNQIKLWRGP7JOCMJAF5NHGTF24A6TID7JVKXNFZATQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.518755006 (12h32m44.033857585s) [noise=true , address=104.36.82.15:4267 , piece_id=DSRR5EHTM75MM7P4MG2YNPHWAE7WHALDCRE4L3YZKKH2BMKRTKAA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.522094892 (12h32m44.037197472s) [noise=true , address=104.36.82.18:4383 , piece_id=XVON7B5A6KIUDIZUJKNMNDQBQ5JNTCORJO7JL4NLPYOIGYU63TXQ, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.523039320 (12h32m44.038141897s) [noise=true , address=104.36.82.43:4856 , piece_id=PIKKNIOZICWM7EJM3NSRIJWPLBXIAP7WNT7RBREAHU6JRXZDCHYA, seg_pos={0 0}, upload=12956]: Uploading piece...
Aug 7 22:12:09.590300484 (12h32m44.105403049s) [upload=12956]: Next returned nil segment
Aug 7 22:12:09.590317010 (12h32m44.105419591s) [upload=12956]: Waiting for error group managing segments...
Aug 7 22:12:09.604186438 (12h32m44.119289008s) [noise=true , address=45.91.116.11:4132 , piece_id=7DYDCKW5NLR22DRUCWIMELTSHM325ESELDERG26BTA2YLWQE2HHA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.606286681 (12h32m44.121389287s) [noise=true , address=104.36.82.59:4326 , piece_id=SHCRSGVR3LRTNV2GEXSWQWSOYPHUOOHDJPYOUALO4M34TZGCHHBA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.620487996 (12h32m44.135590569s) [noise=true , address=104.36.82.61:4386 , piece_id=PNWQU2MPHMRVDXWMMWVNLWGLFQAMEJRLGJXFRCT4M3QYDEG5GSFQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.620594571 (12h32m44.135697142s) [noise=true , address=77.90.31.14:4683 , piece_id=4NNJC6OFHOZNQIKLWRGP7JOCMJAF5NHGTF24A6TID7JVKXNFZATQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.625884604 (12h32m44.140987167s) [noise=true , address=104.36.82.15:4267 , piece_id=DSRR5EHTM75MM7P4MG2YNPHWAE7WHALDCRE4L3YZKKH2BMKRTKAA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.626396996 (12h32m44.141499578s) [noise=true , address=104.36.82.58:4292 , piece_id=PFHGP3WHANRR4CSDD3VN5MBXQ65OV7UGJ5WDUP34AZYQZAMRLU7A, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.629464012 (12h32m44.144566574s) [noise=true , address=104.36.82.5:4844 , piece_id=GDXNIVT4KT5XFLFAR7B7NMHQDCE2OGGJQYHWID5ASAWO2IVNBN7Q, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.643428107 (12h32m44.158530675s) [noise=true , address=104.36.82.14:4248 , piece_id=7ZRBZNDCIDWWONITBOTR3OKLOWVCZ5VCANY6YAQQW6JYSUBUMJZA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.643719221 (12h32m44.15882179s) [noise=true , address=104.36.82.44:4873 , piece_id=GLETUFDVURZ3XPA2OHFTOA7NLKIQSPKF6HVFNPGBOOPU7ICLEOAA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.644290441 (12h32m44.159393003s) [noise=true , address=104.36.82.34:4575 , piece_id=MQE24KHKGXFJPHPKZH4NAI3G2FSOWA42JGN2WEXU47J6ALXEMFOQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.646798488 (12h32m44.161901065s) [noise=true , address=104.36.82.43:4856 , piece_id=PIKKNIOZICWM7EJM3NSRIJWPLBXIAP7WNT7RBREAHU6JRXZDCHYA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.656991689 (12h32m44.172094259s) [noise=true , address=104.36.82.56:4257 , piece_id=6CTH2UDV6ECR5GOBNLG36WD63IHMQTVYH3IUYH33RZYB4ZQ6LQ2A, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:09.835525769 (12h32m44.350628346s) [noise=true , address=79.127.201.215:4357 , piece_id=I2SQGM7NHIUHKNQG6RT3PH3W6P5OKJPP3MFJ65QC7CTN36DUYYJA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:10.167762322 (12h32m44.682864893s) [noise=true , address=64.49.13.154:4934 , piece_id=5T4YXQHU4DJEZCEAHQT3U4P636W5WDGSQCVCB73XQOCJFAV4O2MA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: <nil>
Aug 7 22:12:10.167785966 (12h32m44.682888542s) [seg_pos={0 0}, upload=12956]: Segment reached optimal threshold of 14 pieces.
Aug 7 22:12:10.167971590 (12h32m44.683074167s) [noise=true , address=64.49.13.137:4493 , piece_id=432XMLABWE3DSWXU3B3BTREMG6CP7ILTNU4RJPA6JLHEJNX5POIA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12N91KV4t7yYJvKLFMaXHjByCZpsNgWFnYyZHJE7BqEYBGMLAi9): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168073316 (12h32m44.683175878s) [noise=true , address=104.36.82.18:4383 , piece_id=XVON7B5A6KIUDIZUJKNMNDQBQ5JNTCORJO7JL4NLPYOIGYU63TXQ, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Wtu7qHz3z5gYVPLJzaKKH3orBrtBUEVUeHRUaUe4xJGZv4u4z): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168146785 (12h32m44.683249347s) [noise=true , address=64.49.13.138:4490 , piece_id=RT5AQYMACT6OGCNBPVOJOLYGYQHHND6TVS3FEE52MXZWXZJLCCVA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Jp1vcm8dpppQHEfAtYmM29CDS3mqhojhuVrzkL3PsYHW2q8zA): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168258506 (12h32m44.683361075s) [noise=true , address=79.127.205.235:4670 , piece_id=3GWZPOLPZFZUAJKZ6MOXRTGGE7YDBMNP53WLJXEVOYW2ES5MLZPA, seg_pos={0 0}, upload=12956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JgTfMzsd3rMqQwNim6BoZSiePoLW3gcUtraj84Kn4Szy7aNga): protocol: expected piece hash; context canceled; EOF
Aug 7 22:12:10.168271822 (12h32m44.68337439s) [seg_pos={0 0}, upload=12956]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 22:12:10.168367951 (12h32m44.683470527s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 22:12:10.168372753 (12h32m44.683475329s) [upload=12956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 22:12:11.355196489 (12h32m45.870299059s) [upload=12956]: Done uploading segments...
Aug 7 22:12:11.355214126 (12h32m45.870316692s) [upload=12956]: Upload finished. err: <nil>
Duration: 1.0642352s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
25 out of 60 is failed (threshold=3) [XXXXX.XX...XXX...............................XXXXXXXXXXXXXXX] duration:
30.001484108s
FAIL (2025-08-07 22:22:51 UTC)
operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 877 (12h36m15.180602442s) [upload=13010]: Uploading segments... Aug 7 22:15:40.665590817 (12h36m15.180693396s) [upload=13010]: Got next segment. Inline: true Aug 7 22:15:40.665597143 (12h36m15.18069972s) [upload=13010]: Waiting for error group managing segments... Aug 7 22:15:40.665671744 (12h36m15.180774307s) [upload=13010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:15:40.665678967 (12h36m15.180781528s) [upload=13010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:15:40.665681447 (12h36m15.180784023s) [upload=13010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:16:10.636048401 (12h36m45.151150968s) [upload=13010]: Done uploading segments... Aug 7 22:16:10.636067176 (12h36m45.151169742s) [upload=13010]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:16:19.727995115 (12h36m54.243097692s) [upload=13016]: Starting upload Aug 7 22:16:19.728064428 (12h36m54.243167004s) [upload=13016]: Done starting upload Aug 7 22:16:19.728134488 (12h36m54.243237051s) [upload=13016]: Uploading segments... Aug 7 22:16:19.728182154 (12h36m54.243284736s) [upload=13016]: Got next segment. Inline: true Aug 7 22:16:19.728186380 (12h36m54.243288955s) [upload=13016]: Waiting for error group managing segments... Aug 7 22:16:19.728223320 (12h36m54.243325899s) [upload=13016]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:16:19.728227150 (12h36m54.243329727s) [upload=13016]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:16:19.728229527 (12h36m54.243332103s) [upload=13016]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:16:49.691229995 (12h37m24.206332575s) [upload=13016]: Done uploading segments... Aug 7 22:16:49.691252411 (12h37m24.206354988s) [upload=13016]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:17:13.655336229 (12h37m48.170438797s) [upload=13028]: Starting upload Aug 7 22:17:13.655409916 (12h37m48.170512492s) [upload=13028]: Done starting upload Aug 7 22:17:13.655478262 (12h37m48.17058083s) [upload=13028]: Uploading segments... Aug 7 22:17:13.655548537 (12h37m48.17065113s) [upload=13028]: Got next segment. Inline: true Aug 7 22:17:13.655555256 (12h37m48.170657826s) [upload=13028]: Waiting for error group managing segments... Aug 7 22:17:13.655620901 (12h37m48.170723469s) [upload=13028]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:17:13.655627928 (12h37m48.170730508s) [upload=13028]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:13.655632202 (12h37m48.170734786s) [upload=13028]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:17:43.624980277 (12h38m18.140082854s) [upload=13028]: Done uploading segments... Aug 7 22:17:43.624997725 (12h38m18.140100301s) [upload=13028]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:17:43.747114121 (12h38m18.2622167s) [upload=13037]: Starting upload Aug 7 22:17:43.747186113 (12h38m18.262288692s) [upload=13037]: Done starting upload Aug 7 22:17:43.747260024 (12h38m18.262362591s) [upload=13037]: Uploading segments... Aug 7 22:17:43.747409410 (12h38m18.262511987s) [upload=13037]: Got next segment. Inline: true Aug 7 22:17:43.747414935 (12h38m18.262517512s) [upload=13037]: Waiting for error group managing segments... Aug 7 22:17:43.747457457 (12h38m18.262560036s) [upload=13037]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:17:43.747461997 (12h38m18.262564572s) [upload=13037]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:43.747465159 (12h38m18.262567736s) [upload=13037]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:18:13.715508579 (12h38m48.230611159s) [upload=13037]: Done uploading segments... Aug 7 22:18:13.715531957 (12h38m48.230634536s) [upload=13037]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:18:33.297944141 (12h39m7.813046721s) [upload=13046]: Starting upload Aug 7 22:18:33.298041745 (12h39m7.813144331s) [upload=13046]: Done starting upload Aug 7 22:18:33.298133963 (12h39m7.813236527s) [upload=13046]: Uploading segments... Aug 7 22:18:33.298195648 (12h39m7.813298236s) [upload=13046]: Got next segment. Inline: true Aug 7 22:18:33.298201348 (12h39m7.813303934s) [upload=13046]: Waiting for error group managing segments... Aug 7 22:18:33.298250060 (12h39m7.813352642s) [upload=13046]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:18:33.298255690 (12h39m7.813358273s) [upload=13046]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:18:33.298259809 (12h39m7.813362387s) [upload=13046]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:19:03.266409729 (12h39m37.781512306s) [upload=13046]: Done uploading segments... Aug 7 22:19:03.266458971 (12h39m37.781561549s) [upload=13046]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:19:30.020553557 (12h40m4.535656125s) [upload=13060]: Starting upload Aug 7 22:19:30.020630355 (12h40m4.535732931s) [upload=13060]: Done starting upload Aug 7 22:19:30.020870620 (12h40m4.535973198s) [upload=13060]: Uploading segments... Aug 7 22:19:30.020984540 (12h40m4.536087103s) [upload=13060]: Got next segment. Inline: true Aug 7 22:19:30.020990929 (12h40m4.536093523s) [upload=13060]: Waiting for error group managing segments... Aug 7 22:19:30.021057882 (12h40m4.536160461s) [upload=13060]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:19:30.021063707 (12h40m4.536166287s) [upload=13060]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:19:30.021067070 (12h40m4.536169633s) [upload=13060]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:19:59.990140275 (12h40m34.505242847s) [upload=13060]: Done uploading segments... Aug 7 22:19:59.990166176 (12h40m34.505268763s) [upload=13060]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:14.462825235 (12h40m48.977927811s) [upload=13071]: Starting upload Aug 7 22:20:14.462946370 (12h40m48.978048965s) [upload=13071]: Done starting upload Aug 7 22:20:14.463094242 (12h40m48.978196809s) [upload=13071]: Uploading segments... Aug 7 22:20:14.463162579 (12h40m48.978265164s) [upload=13071]: Got next segment. Inline: true Aug 7 22:20:14.463168150 (12h40m48.978270729s) [upload=13071]: Waiting for error group managing segments... Aug 7 22:20:14.463216881 (12h40m48.978319457s) [upload=13071]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:20:14.463226562 (12h40m48.978329138s) [upload=13071]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:14.463230088 (12h40m48.978332654s) [upload=13071]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:20:44.419309349 (12h41m18.934411917s) [upload=13071]: Done uploading segments... Aug 7 22:20:44.419327995 (12h41m18.934430565s) [upload=13071]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:59.026227228 (12h41m33.541329811s) [upload=13081]: Starting upload Aug 7 22:20:59.026335834 (12h41m33.541438465s) [upload=13081]: Done starting upload Aug 7 22:20:59.026413148 (12h41m33.541515725s) [upload=13081]: Uploading segments... Aug 7 22:20:59.026487957 (12h41m33.541590544s) [upload=13081]: Got next segment. Inline: true Aug 7 22:20:59.026494660 (12h41m33.54159724s) [upload=13081]: Waiting for error group managing segments... Aug 7 22:20:59.026548283 (12h41m33.541650867s) [upload=13081]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:20:59.026554014 (12h41m33.541656591s) [upload=13081]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:59.026557057 (12h41m33.541659621s) [upload=13081]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:21:28.995870666 (12h42m3.510973249s) [upload=13081]: Done uploading segments... Aug 7 22:21:28.995890051 (12h42m3.510992681s) [upload=13081]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:21:29.129769938 (12h42m3.644872515s) [upload=13090]: Starting upload Aug 7 22:21:29.129856429 (12h42m3.644959013s) [upload=13090]: Done starting upload Aug 7 22:21:29.129943698 (12h42m3.645046276s) [upload=13090]: Uploading segments... Aug 7 22:21:29.129985259 (12h42m3.645087845s) [upload=13090]: Got next segment. Inline: true Aug 7 22:21:29.129990476 (12h42m3.645093052s) [upload=13090]: Waiting for error group managing segments... Aug 7 22:21:29.130027566 (12h42m3.645130145s) [upload=13090]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:21:29.130031892 (12h42m3.645134469s) [upload=13090]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:21:29.130035507 (12h42m3.645138082s) [upload=13090]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:21:59.086555047 (12h42m33.601657631s) [upload=13090]: Done uploading segments... Aug 7 22:21:59.086578788 (12h42m33.601681377s) [upload=13090]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:22:21.787894978 (12h42m56.302997579s) [upload=13102]: Starting upload Aug 7 22:22:21.787975806 (12h42m56.303078386s) [upload=13102]: Done starting upload Aug 7 22:22:21.788049720 (12h42m56.303152297s) [upload=13102]: Uploading segments... Aug 7 22:22:21.788092210 (12h42m56.303194795s) [upload=13102]: Got next segment. Inline: true Aug 7 22:22:21.788097559 (12h42m56.303200126s) [upload=13102]: Waiting for error group managing segments... Aug 7 22:22:21.788141372 (12h42m56.303243949s) [upload=13102]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 22:22:21.788146123 (12h42m56.30324876s) [upload=13102]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:21.788149819 (12h42m56.303252395s) [upload=13102]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 22:22:51.758466888 (12h43m26.273569473s) [upload=13102]: Done uploading segments... Aug 7 22:22:51.758489416 (12h43m26.273591984s) [upload=13102]: Upload finished. err: metaclient: context deadline exceeded
Duration: 30.001526983s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
7 out of 60 is failed (threshold=3) [XX..XX................................................X..XX.] duration:
16.690102625s remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-07 22:22:46 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 16.690119196s
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
26 out of 60 is failed (threshold=3) [.XXXXXXXXXXXXXXXXXX.XX.X..X.............................XXXX] duration:
30.001381898s
FAIL (2025-08-07 22:22:44 UTC)
operation: sync2.Copy uplink: encryption: context deadline exceeded
Duration: 30.00140445s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
26 out of 60 is failed (threshold=3) [XXXXXXXXXXXX.XX...X.............................XXXXXXXXXXX.] duration:
28.804983299s
FAIL (2025-08-07 22:22:42 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 28.805007933s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
28 out of 60 is failed (threshold=3) [XXXXXXXX.XX.X..X..X..........................XXXXXXXXXXXXXXX] duration:
30.00050587s
FAIL (2025-08-07 22:22:41 UTC)
operation: Commit uplink: metaclient: context deadline exceeded
Duration: 30.000525498s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
29 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXX.....X..........................XXXXXXXXX] duration:
1m0.001000578s
FAIL (2025-08-07 22:22:37 UTC)
operation: sync2.Copy
======ERROR======
uplink: encryption: context deadline exceeded
======LOGS======
936 (12h31m52.807000513s) [noise=true , address=104.36.82.60:4342 , piece_id=5LZ247YTSW7ABTMGS75QYX5ATR5O47XU7UAOSNH2DYHXSE36FN2Q, seg_pos={0 0}, upload=12943]: Done uploading piece. err: <nil>
Aug 7 22:11:18.291922152 (12h31m52.80702473s) [seg_pos={0 0}, upload=12943]: Segment reached optimal threshold of 14 pieces.
Aug 7 22:11:18.292352474 (12h31m52.807455054s) [noise=true , address=104.36.82.35:4591 , piece_id=KUKTQY554VFN5NOCL7L4W3TJXLSPOZYECLMZGJWNQC26XH5W6P5A, seg_pos={0 0}, upload=12943]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a453d6xSb26epbgqPuco3UPTf9yzoxBeKToCUMNiNHUddAq55): protocol: expected piece hash; context canceled; EOF
Aug 7 22:11:18.292749416 (12h31m52.807851996s) [noise=true , address=104.36.82.61:4383 , piece_id=YTMXPHQSLO6AHZCGEA3KR5YUGMHH3UPGEX26JNQ6MXSIKW6AJUMA, seg_pos={0 0}, upload=12943]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12N24q6Li2xavkE61KntNxDPwQzTVHaG4LxyEqFUT1hhGYNQ9mp): protocol: write tcp 159.223.143.39:45544->104.36.82.61:4383: use of closed network connection; write tcp 159.223.143.39:45544->104.36.82.61:4383: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:45544->104.36.82.61:4383: use of closed network connection
Aug 7 22:11:18.293007428 (12h31m52.808110005s) [noise=true , address=64.49.13.26:4950 , piece_id=DW4TXDRT7ZGXOOU3RKT4ACLWMYFAGOPD6Q2HV7TFESGXSOC4DE6A, seg_pos={0 0}, upload=12943]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14Tv25fLSqTnZQiDzrYqKCSbdt6ep3upgqD2Se8QSRuQFynkSN): protocol: read tcp 159.223.143.39:34280->64.49.13.26:4950: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 7 22:11:18.293070269 (12h31m52.808172846s) [noise=true , address=104.36.82.45:4909 , piece_id=UYJL3JPM7VCPGSXZ5J263PRO5K75MLZIWQTA3QCPQEQNXN3LVTNQ, seg_pos={0 0}, upload=12943]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14Pqp5cbhmd44QnikAuD5TgsKFDR3gB2qtLuVSroaWVv4U2cDh): protocol: read tcp 159.223.143.39:46438->104.36.82.45:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 7 22:11:18.293086568 (12h31m52.808189151s) [seg_pos={0 0}, upload=12943]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 22:11:18.293166574 (12h31m52.808269155s) [upload=12943]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 22:11:18.293172005 (12h31m52.808274588s) [upload=12943]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 22:14:08.727892812 (12h34m43.242995378s) [upload=12943]: Done uploading segments...
Aug 7 22:14:08.727919062 (12h34m43.243021627s) [upload=12943]: Upload finished. err: metaclient: context deadline exceeded
Aug 7 22:14:08.820786996 (12h34m43.335889564s) [upload=12987]: Starting upload
Aug 7 22:14:08.820869466 (12h34m43.335972332s) [upload=12987]: Done starting upload
Aug 7 22:14:08.820990812 (12h34m43.336093389s) [upload=12987]: Uploading segments...
Aug 7 22:14:08.821180945 (12h34m43.336283523s) [upload=12987]: Got next segment. Inline: false
Aug 7 22:14:08.821227735 (12h34m43.336330315s) [upload=12987]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 22:14:08.821233790 (12h34m43.336336478s) [upload=12987]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 22:15:08.788378642 (12h35m43.303481218s) [upload=12987]: Next segment err: context deadline exceeded
Aug 7 22:15:08.788567212 (12h35m43.303669789s) [upload=12987]: Done uploading segments...
Aug 7 22:15:08.788593806 (12h35m43.303696388s) [upload=12987]: Upload finished. err: context deadline exceeded
Aug 7 22:15:22.518494042 (12h35m57.033596616s) [upload=13004]: Starting upload
Aug 7 22:15:22.518605977 (12h35m57.033708558s) [upload=13004]: Done starting upload
Aug 7 22:15:22.518927572 (12h35m57.034030143s) [upload=13004]: Uploading segments...
Aug 7 22:15:22.519083330 (12h35m57.034185915s) [upload=13004]: Got next segment. Inline: false
Aug 7 22:15:22.519139330 (12h35m57.034241894s) [upload=13004]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 22:15:22.519145674 (12h35m57.034248259s) [upload=13004]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 22:16:22.486086322 (12h36m57.001188918s) [upload=13004]: Next segment err: context deadline exceeded
Aug 7 22:16:22.486315849 (12h36m57.00141843s) [upload=13004]: Done uploading segments...
Aug 7 22:16:22.486353471 (12h36m57.001456038s) [upload=13004]: Upload finished. err: context deadline exceeded
Aug 7 22:16:26.977501497 (12h37m1.492604079s) [upload=13017]: Starting upload
Aug 7 22:16:26.977634110 (12h37m1.492736693s) [upload=13017]: Done starting upload
Aug 7 22:16:26.977724277 (12h37m1.492826848s) [upload=13017]: Uploading segments...
Aug 7 22:16:26.978074717 (12h37m1.493177287s) [upload=13017]: Got next segment. Inline: false
Aug 7 22:16:26.978159060 (12h37m1.493261638s) [upload=13017]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 22:16:26.978164291 (12h37m1.493266867s) [upload=13017]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 22:17:26.948529899 (12h38m1.463632483s) [upload=13017]: Next segment err: context deadline exceeded
Aug 7 22:17:26.948553603 (12h38m1.463656182s) [upload=13017]: Done uploading segments...
Aug 7 22:17:26.948580927 (12h38m1.463683497s) [upload=13017]: Upload finished. err: context deadline exceeded
Aug 7 22:17:27.068460198 (12h38m1.583562778s) [upload=13032]: Starting upload
Aug 7 22:17:27.068564049 (12h38m1.583666643s) [upload=13032]: Done starting upload
Aug 7 22:17:27.068889168 (12h38m1.583991745s) [upload=13032]: Uploading segments...
Aug 7 22:17:27.069219219 (12h38m1.584321786s) [upload=13032]: Got next segment. Inline: false
Aug 7 22:17:27.069277883 (12h38m1.584380461s) [upload=13032]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 22:17:27.069305785 (12h38m1.58440837s) [upload=13032]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 22:18:27.039567178 (12h39m1.554669767s) [upload=13032]: Next segment err: context deadline exceeded
Aug 7 22:18:27.039666174 (12h39m1.554768736s) [upload=13032]: Done uploading segments...
Aug 7 22:18:27.039688215 (12h39m1.554790778s) [upload=13032]: Upload finished. err: context deadline exceeded
Aug 7 22:18:36.777819622 (12h39m11.292922206s) [upload=13047]: Starting upload
Aug 7 22:18:36.777959930 (12h39m11.293062515s) [upload=13047]: Done starting upload
Aug 7 22:18:36.778221018 (12h39m11.29332361s) [upload=13047]: Uploading segments...
Aug 7 22:18:36.778469733 (12h39m11.293572319s) [upload=13047]: Got next segment. Inline: false
Aug 7 22:18:36.778552648 (12h39m11.293655234s) [upload=13047]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 22:18:36.778559731 (12h39m11.293662318s) [upload=13047]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 22:19:36.748208363 (12h40m11.263310949s) [upload=13047]: Next segment err: context deadline exceeded
Aug 7 22:19:36.748420878 (12h40m11.263523459s) [upload=13047]: Done uploading segments...
Aug 7 22:19:36.748450879 (12h40m11.26355346s) [upload=13047]: Upload finished. err: context deadline exceeded
Aug 7 22:19:36.874412908 (12h40m11.389515476s) [upload=13064]: Starting upload
Aug 7 22:19:36.874484846 (12h40m11.389587414s) [upload=13064]: Done starting upload
Aug 7 22:19:36.874815576 (12h40m11.389918144s) [upload=13064]: Uploading segments...
Aug 7 22:19:36.874886597 (12h40m11.389989181s) [upload=13064]: Got next segment. Inline: false
Aug 7 22:19:36.874921953 (12h40m11.390024535s) [upload=13064]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 22:19:36.874930610 (12h40m11.390033195s) [upload=13064]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 22:20:36.842769930 (12h41m11.357872515s) [upload=13064]: Next segment err: context deadline exceeded
Aug 7 22:20:36.842833283 (12h41m11.357935846s) [upload=13064]: Done uploading segments...
Aug 7 22:20:36.842847394 (12h41m11.357949957s) [upload=13064]: Upload finished. err: context deadline exceeded
Aug 7 22:20:36.950394779 (12h41m11.465497359s) [upload=13076]: Starting upload
Aug 7 22:20:36.950472520 (12h41m11.4655751s) [upload=13076]: Done starting upload
Aug 7 22:20:36.950562858 (12h41m11.465665423s) [upload=13076]: Uploading segments...
Aug 7 22:20:36.950771505 (12h41m11.465874083s) [upload=13076]: Got next segment. Inline: false
Aug 7 22:20:36.950816148 (12h41m11.465918724s) [upload=13076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 22:20:36.950820956 (12h41m11.465923532s) [upload=13076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 22:21:36.921505604 (12h42m11.43660819s) [upload=13076]: Next segment err: context deadline exceeded
Aug 7 22:21:36.921531258 (12h42m11.436633847s) [upload=13076]: Done uploading segments...
Aug 7 22:21:36.921563121 (12h42m11.436665701s) [upload=13076]: Upload finished. err: context deadline exceeded
Aug 7 22:21:37.037517602 (12h42m11.552620183s) [upload=13092]: Starting upload
Aug 7 22:21:37.037616359 (12h42m11.552718948s) [upload=13092]: Done starting upload
Aug 7 22:21:37.037766636 (12h42m11.552869214s) [upload=13092]: Uploading segments...
Aug 7 22:21:37.038118108 (12h42m11.553220684s) [upload=13092]: Got next segment. Inline: false
Aug 7 22:21:37.038164037 (12h42m11.553266616s) [upload=13092]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 22:21:37.038168950 (12h42m11.553271526s) [upload=13092]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 22:22:37.006777446 (12h43m11.521880035s) [upload=13092]: Next segment err: context deadline exceeded
Aug 7 22:22:37.007012864 (12h43m11.522115449s) [upload=13092]: Done uploading segments...
Aug 7 22:22:37.007106471 (12h43m11.522209054s) [upload=13092]: Upload finished. err: context deadline exceeded
Duration: 1m0.001036374s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
21 out of 60 is failed (threshold=3) [XXX.X....X.X.................................XXXXXXXXXXXXXXX] duration:
30.000772399s
FAIL (2025-08-07 22:22:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000811596s
edge
gateway
us-select-1
|