Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [..XXX..X.X.X.....................................X..........] duration: 603.151245ms FAIL (2025-08-07 23:57:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 603.179935ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [X.X....XX..........................................X..X....X] duration: 30.001123902s FAIL (2025-08-07 23:57:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001148901s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 25.961463ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 23:57:54 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 25.985488ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 11 out of 60 is failed (threshold=3) [XXXX.X.XX...........................................X..XXX..] duration: 230.904399ms FAIL (2025-08-07 23:57:43 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 230.931167ms core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 30.960335ms target: https://us1.storj.io/login FAIL (2025-08-07 23:57:38 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 30.98503ms core satellite us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 117.092915ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:57:30 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 117.123682ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [XXXXX.X.XX...........................................X..XXX.] duration: 260.910111ms FAIL (2025-08-07 23:57:27 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 260.927792ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [........X...........................................XXX.X...] duration: 593.759812ms FAIL (2025-08-07 23:57:25 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== (14h15m11.016995473s) [upload=14665]: Uploading segments... Aug 7 23:54:36.502140541 (14h15m11.017243126s) [upload=14665]: Got next segment. Inline: false Aug 7 23:54:36.502208316 (14h15m11.017310893s) [upload=14665]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:36.502213415 (14h15m11.017315993s) [upload=14665]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:54:36.540470046 (14h15m11.055572633s) [seg_pos={0 0}, upload=14665]: Begin upload segment... Aug 7 23:54:36.540542869 (14h15m11.055645435s) [seg_pos={0 0}, upload=14665]: Done begin upload segment. Aug 7 23:54:36.540586398 (14h15m11.055688977s) [noise=true , address=77.90.31.10:4520 , piece_id=UXBIKFQ4KOGFTCR73EVF7DBFLAZXOJPPHZUVL4BTJIONLHHQOF3Q, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.540599057 (14h15m11.055701646s) [noise=true , address=104.36.82.7:4909 , piece_id=ISAGUZRBSZZOI5D3XK4N2VBH6CGHWALHK7DUKKDKYTA3PH54WE7A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.541179792 (14h15m11.056282359s) [noise=true , address=45.91.116.6:4975 , piece_id=GSG54APU3E2NKOOHZDOELFVKDEWZPDNYAC6GUW63N2WHS54AHJ5A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.541219031 (14h15m11.056321617s) [noise=true , address=45.91.116.7:4010 , piece_id=HIHBEZ2SFGNUPDJSUPBGYJYIQE5LT5U3KIOWJWTKQWNWHSMB2R6Q, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.541591033 (14h15m11.056693604s) [noise=true , address=104.36.82.5:4837 , piece_id=FJIF5DAJKKHVPDI4FJQFRIY66V6LQXEN7GVMZLCWFPG35QJGSO3Q, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.541808787 (14h15m11.056911375s) [noise=true , address=104.36.82.13:4185 , piece_id=DY4ZRJG7XBEMN5JTETQR5I6HSLLQNIT3R6ZS7CA5ZBW6QAQVGXSA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.542414303 (14h15m11.057516892s) [noise=true , address=104.36.82.16:4294 , piece_id=UHSQZJSGHU3UWAEEOTG5P4NV2YGTGKYYJU7CQ7UC4J6WDUJPUGJA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.542588591 (14h15m11.057691157s) [noise=true , address=104.36.82.4:4802 , piece_id=QHJY5W7S3OB2TAIBARTKQTB6HHBRGKPZP2CM7F4V7NYZQLZZSQ7A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.542987544 (14h15m11.058090129s) [noise=true , address=104.36.82.8:4972 , piece_id=F3E53EQXBSUIZDZV5L5CBCYYVE5BOEMSTVEZZMEI7NKKHHKPD2LQ, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.543563707 (14h15m11.058666294s) [noise=true , address=104.36.82.8:4971 , piece_id=HC2FEV5IWEXHECRHUAMAF7J7J4AWYZRTLRD5NUREB7SBCUEA4VGQ, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.543788500 (14h15m11.058891086s) [noise=true , address=104.36.82.6:4876 , piece_id=BABG2QF67NQHDH62KKGR2FLZRTF5TWGBNQ5D2PKQL6QJURYIROGA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.544354038 (14h15m11.059456606s) [noise=true , address=77.90.31.12:4589 , piece_id=L4B3OU3E72GCPP4SZLWPJ2B3LNS6GSFX3FFWQBCVBHL3JGHR7MGA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.544746164 (14h15m11.059848744s) [noise=true , address=104.36.82.2:4730 , piece_id=KY5B7UDNU2S2ZFHCVWZAPJV4QHEMCT4RKKJLOM3SXKTC22HVGKVA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.545099366 (14h15m11.060201945s) [noise=true , address=77.90.31.19:4841 , piece_id=QV43X4G6B3IXCVDVAJYSGXUUI6LCROUL3IHGNES5JTUEPGKNFA5A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.545536838 (14h15m11.060639427s) [noise=true , address=77.90.31.14:4663 , piece_id=6OHGW5KCETKDQUXWUUD7RSP6JXHCKTSCPAA22E3QP46OWHE4A2FQ, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.546110762 (14h15m11.061213352s) [noise=true , address=77.90.31.17:4771 , piece_id=EH7SHR3OQ3ODCJ2QKQBVROK3BNG6E4B6PKMTNQMDUA4RRZAGAZ2A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.648423840 (14h15m11.16352641s) [upload=14665]: Next returned nil segment Aug 7 23:54:36.648440535 (14h15m11.163543117s) [upload=14665]: Waiting for error group managing segments... Aug 7 23:54:36.654558819 (14h15m11.169661399s) [noise=true , address=45.91.116.7:4010 , piece_id=HIHBEZ2SFGNUPDJSUPBGYJYIQE5LT5U3KIOWJWTKQWNWHSMB2R6Q, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.655196697 (14h15m11.170299261s) [noise=true , address=77.90.31.10:4520 , piece_id=UXBIKFQ4KOGFTCR73EVF7DBFLAZXOJPPHZUVL4BTJIONLHHQOF3Q, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.658908365 (14h15m11.174010932s) [noise=true , address=77.90.31.17:4771 , piece_id=EH7SHR3OQ3ODCJ2QKQBVROK3BNG6E4B6PKMTNQMDUA4RRZAGAZ2A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.659089278 (14h15m11.174191848s) [noise=true , address=45.91.116.6:4975 , piece_id=GSG54APU3E2NKOOHZDOELFVKDEWZPDNYAC6GUW63N2WHS54AHJ5A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.660774579 (14h15m11.175877148s) [noise=true , address=77.90.31.14:4663 , piece_id=6OHGW5KCETKDQUXWUUD7RSP6JXHCKTSCPAA22E3QP46OWHE4A2FQ, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.661054908 (14h15m11.176157476s) [noise=true , address=77.90.31.12:4589 , piece_id=L4B3OU3E72GCPP4SZLWPJ2B3LNS6GSFX3FFWQBCVBHL3JGHR7MGA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.662842221 (14h15m11.17794479s) [noise=true , address=77.90.31.19:4841 , piece_id=QV43X4G6B3IXCVDVAJYSGXUUI6LCROUL3IHGNES5JTUEPGKNFA5A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.663362200 (14h15m11.178464776s) [noise=true , address=104.36.82.6:4876 , piece_id=BABG2QF67NQHDH62KKGR2FLZRTF5TWGBNQ5D2PKQL6QJURYIROGA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.666343769 (14h15m11.181446338s) [noise=true , address=104.36.82.16:4294 , piece_id=UHSQZJSGHU3UWAEEOTG5P4NV2YGTGKYYJU7CQ7UC4J6WDUJPUGJA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.675326798 (14h15m11.190429377s) [noise=true , address=104.36.82.2:4730 , piece_id=KY5B7UDNU2S2ZFHCVWZAPJV4QHEMCT4RKKJLOM3SXKTC22HVGKVA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.678428557 (14h15m11.193531125s) [noise=true , address=104.36.82.4:4802 , piece_id=QHJY5W7S3OB2TAIBARTKQTB6HHBRGKPZP2CM7F4V7NYZQLZZSQ7A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.685873433 (14h15m11.200976001s) [noise=true , address=104.36.82.8:4972 , piece_id=F3E53EQXBSUIZDZV5L5CBCYYVE5BOEMSTVEZZMEI7NKKHHKPD2LQ, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.693259479 (14h15m11.208362058s) [noise=true , address=104.36.82.8:4971 , piece_id=HC2FEV5IWEXHECRHUAMAF7J7J4AWYZRTLRD5NUREB7SBCUEA4VGQ, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.693284102 (14h15m11.208386678s) [seg_pos={0 0}, upload=14665]: Segment reached optimal threshold of 13 pieces. Aug 7 23:54:36.693493083 (14h15m11.20859566s) [noise=true , address=104.36.82.7:4909 , piece_id=ISAGUZRBSZZOI5D3XK4N2VBH6CGHWALHK7DUKKDKYTA3PH54WE7A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF Aug 7 23:54:36.693795135 (14h15m11.208897711s) [noise=true , address=104.36.82.5:4837 , piece_id=FJIF5DAJKKHVPDI4FJQFRIY66V6LQXEN7GVMZLCWFPG35QJGSO3Q, seg_pos={0 0}, upload=14665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:55994->104.36.82.5:4837: use of closed network connection; write tcp 159.223.143.39:55994->104.36.82.5:4837: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:55994->104.36.82.5:4837: use of closed network connection Aug 7 23:54:36.693865018 (14h15m11.208967588s) [noise=true , address=104.36.82.13:4185 , piece_id=DY4ZRJG7XBEMN5JTETQR5I6HSLLQNIT3R6ZS7CA5ZBW6QAQVGXSA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: read tcp 159.223.143.39:52730->104.36.82.13:4185: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:54:36.693875333 (14h15m11.208977898s) [seg_pos={0 0}, upload=14665]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:54:36.693954210 (14h15m11.209056786s) [upload=14665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:54:36.693959007 (14h15m11.209061583s) [upload=14665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:36.756106678 (14h15m11.271209244s) [upload=14665]: Done uploading segments... Aug 7 23:54:36.756123317 (14h15m11.271225887s) [upload=14665]: Upload finished. err: <nil> Aug 7 23:55:38.250097734 (14h16m12.765200314s) [upload=14681]: Starting upload Aug 7 23:55:38.250192737 (14h16m12.765295302s) [upload=14681]: Done starting upload Aug 7 23:55:38.250341846 (14h16m12.765444411s) [upload=14681]: Uploading segments... Aug 7 23:55:38.250743926 (14h16m12.76584651s) [upload=14681]: Got next segment. Inline: false Aug 7 23:55:38.250809082 (14h16m12.765911663s) [upload=14681]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:38.250814071 (14h16m12.765916648s) [upload=14681]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:56:08.219284519 (14h16m42.734387085s) [upload=14681]: Next segment err: context deadline exceeded Aug 7 23:56:08.219364566 (14h16m42.734467135s) [upload=14681]: Done uploading segments... Aug 7 23:56:08.219382617 (14h16m42.734485184s) [upload=14681]: Upload finished. err: context deadline exceeded Duration: 593.785272ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 8 out of 60 is failed (threshold=3) [.XX..X...X..............................................XXXX] duration: 1m0.002213132s FAIL (2025-08-07 23:57:22 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== r: <nil> Aug 7 23:48:32.733662762 (14h9m7.248765339s) [upload=14555]: Starting upload Aug 7 23:48:32.733764614 (14h9m7.248867195s) [upload=14555]: Done starting upload Aug 7 23:48:32.733839718 (14h9m7.248942295s) [upload=14555]: Uploading segments... Aug 7 23:48:32.733888071 (14h9m7.248990659s) [upload=14555]: Got next segment. Inline: true Aug 7 23:48:32.733893767 (14h9m7.248996349s) [upload=14555]: Waiting for error group managing segments... Aug 7 23:48:32.733943638 (14h9m7.249046215s) [upload=14555]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:32.733949815 (14h9m7.249052399s) [upload=14555]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:32.733954110 (14h9m7.24905669s) [upload=14555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:32.809665685 (14h9m7.324768252s) [upload=14555]: Done uploading segments... Aug 7 23:48:32.809699832 (14h9m7.324802408s) [upload=14555]: Upload finished. err: <nil> Aug 7 23:49:43.381878590 (14h10m17.896981166s) [upload=14573]: Starting upload Aug 7 23:49:43.382020923 (14h10m17.897123495s) [upload=14573]: Done starting upload Aug 7 23:49:43.382128913 (14h10m17.897231484s) [upload=14573]: Uploading segments... Aug 7 23:49:43.382204063 (14h10m17.897306641s) [upload=14573]: Got next segment. Inline: true Aug 7 23:49:43.382211147 (14h10m17.897313718s) [upload=14573]: Waiting for error group managing segments... Aug 7 23:49:43.382287220 (14h10m17.897389786s) [upload=14573]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:43.382292849 (14h10m17.897395421s) [upload=14573]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:43.382297255 (14h10m17.897399833s) [upload=14573]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:43.532367962 (14h10m18.047470542s) [upload=14573]: Done uploading segments... Aug 7 23:49:43.532399853 (14h10m18.047502429s) [upload=14573]: Upload finished. err: <nil> Aug 7 23:50:25.794985700 (14h11m0.310088267s) [upload=14588]: Starting upload Aug 7 23:50:25.795091135 (14h11m0.310193704s) [upload=14588]: Done starting upload Aug 7 23:50:25.795192505 (14h11m0.310295073s) [upload=14588]: Uploading segments... Aug 7 23:50:25.795250709 (14h11m0.310353286s) [upload=14588]: Got next segment. Inline: true Aug 7 23:50:25.795256599 (14h11m0.310359177s) [upload=14588]: Waiting for error group managing segments... Aug 7 23:50:25.795298950 (14h11m0.310401527s) [upload=14588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:25.795303481 (14h11m0.310406058s) [upload=14588]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:25.795306901 (14h11m0.310409476s) [upload=14588]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:25.886991014 (14h11m0.402093596s) [upload=14588]: Done uploading segments... Aug 7 23:50:25.887011651 (14h11m0.402114228s) [upload=14588]: Upload finished. err: <nil> Aug 7 23:51:02.166684989 (14h11m36.681787571s) [upload=14598]: Starting upload Aug 7 23:51:02.166793315 (14h11m36.681895904s) [upload=14598]: Done starting upload Aug 7 23:51:02.167617113 (14h11m36.682719692s) [upload=14598]: Uploading segments... Aug 7 23:51:02.167688699 (14h11m36.682791288s) [upload=14598]: Got next segment. Inline: true Aug 7 23:51:02.167696965 (14h11m36.68279955s) [upload=14598]: Waiting for error group managing segments... Aug 7 23:51:02.167768986 (14h11m36.682871569s) [upload=14598]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:02.167776149 (14h11m36.682878967s) [upload=14598]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:02.167781308 (14h11m36.682883888s) [upload=14598]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:02.249738740 (14h11m36.764841311s) [upload=14598]: Done uploading segments... Aug 7 23:51:02.249754857 (14h11m36.764857435s) [upload=14598]: Upload finished. err: <nil> Aug 7 23:51:48.536453050 (14h12m23.051555623s) [upload=14610]: Starting upload Aug 7 23:51:48.536534615 (14h12m23.051637193s) [upload=14610]: Done starting upload Aug 7 23:51:48.536600634 (14h12m23.051703211s) [upload=14610]: Uploading segments... Aug 7 23:51:48.536652707 (14h12m23.051755293s) [upload=14610]: Got next segment. Inline: true Aug 7 23:51:48.536658284 (14h12m23.051760862s) [upload=14610]: Waiting for error group managing segments... Aug 7 23:51:48.536705902 (14h12m23.051808486s) [upload=14610]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:48.536710850 (14h12m23.051813429s) [upload=14610]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:48.536714377 (14h12m23.051816954s) [upload=14610]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:48.601979182 (14h12m23.117081769s) [upload=14610]: Done uploading segments... Aug 7 23:51:48.602001648 (14h12m23.117104229s) [upload=14610]: Upload finished. err: <nil> Aug 7 23:52:04.519337006 (14h12m39.034439591s) [upload=14619]: Starting upload Aug 7 23:52:04.519460694 (14h12m39.034563258s) [upload=14619]: Done starting upload Aug 7 23:52:04.519569621 (14h12m39.034672186s) [upload=14619]: Uploading segments... Aug 7 23:52:04.519675504 (14h12m39.034778091s) [upload=14619]: Got next segment. Inline: true Aug 7 23:52:04.519701072 (14h12m39.034803649s) [upload=14619]: Waiting for error group managing segments... Aug 7 23:52:04.519755555 (14h12m39.034858131s) [upload=14619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:04.519760650 (14h12m39.034863226s) [upload=14619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:04.519777078 (14h12m39.034879654s) [upload=14619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:04.588422656 (14h12m39.103525241s) [upload=14619]: Done uploading segments... Aug 7 23:52:04.588444719 (14h12m39.103547297s) [upload=14619]: Upload finished. err: <nil> Aug 7 23:53:08.147087231 (14h13m42.662189804s) [upload=14641]: Starting upload Aug 7 23:53:08.147181411 (14h13m42.662283996s) [upload=14641]: Done starting upload Aug 7 23:53:08.147281202 (14h13m42.66238377s) [upload=14641]: Uploading segments... Aug 7 23:53:08.147322830 (14h13m42.662425415s) [upload=14641]: Got next segment. Inline: true Aug 7 23:53:08.147327701 (14h13m42.662430277s) [upload=14641]: Waiting for error group managing segments... Aug 7 23:53:08.147455761 (14h13m42.66255833s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:08.147460709 (14h13m42.662563285s) [upload=14641]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:08.147463890 (14h13m42.662566457s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:08.116969456 (14h14m42.632072039s) [upload=14641]: Done uploading segments... Aug 7 23:54:08.117007036 (14h14m42.632109618s) [upload=14641]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:22.831162336 (14h14m57.346264913s) [upload=14661]: Starting upload Aug 7 23:54:22.831234431 (14h14m57.346337008s) [upload=14661]: Done starting upload Aug 7 23:54:22.831306794 (14h14m57.346409362s) [upload=14661]: Uploading segments... Aug 7 23:54:22.831343257 (14h14m57.346445842s) [upload=14661]: Got next segment. Inline: true Aug 7 23:54:22.831347808 (14h14m57.346450375s) [upload=14661]: Waiting for error group managing segments... Aug 7 23:54:22.831399492 (14h14m57.346502071s) [upload=14661]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:22.831404349 (14h14m57.346506915s) [upload=14661]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:22.831407696 (14h14m57.346510265s) [upload=14661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:22.799123829 (14h15m57.314226409s) [upload=14661]: Done uploading segments... Aug 7 23:55:22.799154191 (14h15m57.314256772s) [upload=14661]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:22.918485444 (14h15m57.433588022s) [upload=14677]: Starting upload Aug 7 23:55:22.918566621 (14h15m57.433672141s) [upload=14677]: Done starting upload Aug 7 23:55:22.918647605 (14h15m57.433750172s) [upload=14677]: Uploading segments... Aug 7 23:55:22.918705315 (14h15m57.4338079s) [upload=14677]: Got next segment. Inline: true Aug 7 23:55:22.918710403 (14h15m57.43381298s) [upload=14677]: Waiting for error group managing segments... Aug 7 23:55:22.918764176 (14h15m57.433866753s) [upload=14677]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:22.918769239 (14h15m57.433871815s) [upload=14677]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:22.918772352 (14h15m57.433874928s) [upload=14677]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:22.891103920 (14h16m57.406206502s) [upload=14677]: Done uploading segments... Aug 7 23:56:22.891142892 (14h16m57.406245471s) [upload=14677]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:23.012512156 (14h16m57.527614733s) [upload=14690]: Starting upload Aug 7 23:56:23.012570683 (14h16m57.527673265s) [upload=14690]: Done starting upload Aug 7 23:56:23.012632890 (14h16m57.527735455s) [upload=14690]: Uploading segments... Aug 7 23:56:23.012675514 (14h16m57.527778093s) [upload=14690]: Got next segment. Inline: true Aug 7 23:56:23.012680516 (14h16m57.527783092s) [upload=14690]: Waiting for error group managing segments... Aug 7 23:56:23.012734720 (14h16m57.527837295s) [upload=14690]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:23.012739356 (14h16m57.527841923s) [upload=14690]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:23.012742484 (14h16m57.527845059s) [upload=14690]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:22.981126119 (14h17m57.496228686s) [upload=14690]: Done uploading segments... Aug 7 23:57:22.981169317 (14h17m57.496271889s) [upload=14690]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.002228558s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [.X.X....XX..........................................X..X....] duration: 337.73256ms FAIL (2025-08-07 23:57:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 337.77755ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 11 out of 60 is failed (threshold=3) [.....XX.XXXX...X.X......................................X.XX] duration: 1m0.001395361s FAIL (2025-08-07 23:57:11 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== LSW5OYVOAT43EV525LGTTZA5NOVUXOHGDFGVZQ, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.934331494 (14h15m14.449434079s) [noise=true , address=77.90.31.17:4789 , piece_id=UAPJNIIAY4C72FKIIGLN7J4H3DHMR64YCY53T5KP2BNAODYYYNTQ, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.934339011 (14h15m14.449441596s) [noise=true , address=104.36.82.23:4447 , piece_id=63VNSCHK7YTAW5X5UWOVUORPY4Z55IJEHYI43LYZMQNINP2VG6CA, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.934712713 (14h15m14.449815294s) [noise=true , address=104.36.82.47:4537 , piece_id=CHRH43QZTM357GGR2EWYK4P5N5POLUQQ6TA67VYEJLWQ75A6YXYQ, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.934726332 (14h15m14.449828913s) [noise=true , address=104.36.82.23:4475 , piece_id=ZKLJCXQBVGOJZSHHAZITBWLTIW3XSA3EXJ5IU35Q3AFGOVUSZMLQ, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.935125663 (14h15m14.450228251s) [noise=true , address=104.36.82.35:4608 , piece_id=EUMFYK2C6SLU5P3L6FBUCWPGBEB27GXJJXU6OBM5DXHUYN3C3NEA, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.935141576 (14h15m14.450244165s) [noise=true , address=104.36.82.5:4850 , piece_id=RVQ3JQ3WI2QTVXI5UQHRCQPIHJB6XBE3XVKBULUEH2JEOK4FZENA, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.935504875 (14h15m14.450607458s) [noise=true , address=104.36.82.9:4040 , piece_id=IT2Y4ZC5F5LBQSYOHZ3YAS5GMMIR4GUFTGJQWOM2ZSNZZ7QR6LJA, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.935516825 (14h15m14.450619411s) [noise=true , address=64.49.13.140:4505 , piece_id=VP5P42UVQ3VAROWEWCUDCQBSAPFI5OYX2XYAYUJWSE7WJIEQVJ6Q, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.935881542 (14h15m14.450984124s) [noise=true , address=104.36.82.10:4097 , piece_id=XL72INQPSMGOCIBWNA2GYUOKLLQKVDAQ27NX5UPZ7Y2WHXIT2SGA, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.935889552 (14h15m14.450992138s) [noise=true , address=45.91.116.7:4040 , piece_id=6NXPRIX6B6LBNGEL7JEFMPET53H5R3A3QNOZRAKDXJCLZPX6KG2A, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.936356864 (14h15m14.451459448s) [noise=true , address=104.36.82.46:4508 , piece_id=HG7TGN3Q5SETJRBS6GTVYJDN5EM7E47XBBLF57PD4TKKNCZCCKIQ, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.938275027 (14h15m14.453377604s) [noise=true , address=45.91.116.4:4966 , piece_id=ZEJLCOYGPIFGTYESTSGTMBW2A5EYKI2BAW5CBM7LWUX2ZVZ2OH7A, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.938941485 (14h15m14.454044056s) [noise=true , address=104.36.82.45:4915 , piece_id=E6ZVUDFXRP5FRVOAAQN2XF2MGVSITTCHGU5X5UVZFZZFLAUF4KNQ, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.939612177 (14h15m14.454714763s) [noise=true , address=104.36.82.18:4370 , piece_id=NTCE3JDSCC5U3TRTIQ7ZWFVR3RE2JQL77GIH23PQWXHXIW2E3TTQ, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.939819372 (14h15m14.454921942s) [noise=true , address=45.91.116.8:4059 , piece_id=ALAED7ISKWGLECJEUYVLJOZNNCIVNHUEBYNMBJWALWEJGP2GRCPQ, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:40.038236225 (14h15m14.553338789s) [upload=14660]: Next returned nil segment Aug 7 23:54:40.038258035 (14h15m14.553360621s) [upload=14660]: Waiting for error group managing segments... Aug 7 23:54:40.048242091 (14h15m14.56334466s) [noise=true , address=104.36.82.23:4447 , piece_id=63VNSCHK7YTAW5X5UWOVUORPY4Z55IJEHYI43LYZMQNINP2VG6CA, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.049077306 (14h15m14.564179883s) [noise=true , address=104.36.82.35:4608 , piece_id=EUMFYK2C6SLU5P3L6FBUCWPGBEB27GXJJXU6OBM5DXHUYN3C3NEA, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.051719279 (14h15m14.566821849s) [noise=true , address=104.36.82.18:4370 , piece_id=NTCE3JDSCC5U3TRTIQ7ZWFVR3RE2JQL77GIH23PQWXHXIW2E3TTQ, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.052612115 (14h15m14.567714685s) [noise=true , address=45.91.116.4:4966 , piece_id=ZEJLCOYGPIFGTYESTSGTMBW2A5EYKI2BAW5CBM7LWUX2ZVZ2OH7A, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.060218512 (14h15m14.575321102s) [noise=true , address=45.91.116.8:4059 , piece_id=ALAED7ISKWGLECJEUYVLJOZNNCIVNHUEBYNMBJWALWEJGP2GRCPQ, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.065281928 (14h15m14.580384497s) [noise=true , address=45.91.116.7:4040 , piece_id=6NXPRIX6B6LBNGEL7JEFMPET53H5R3A3QNOZRAKDXJCLZPX6KG2A, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.066022729 (14h15m14.581125298s) [noise=true , address=104.36.82.23:4475 , piece_id=ZKLJCXQBVGOJZSHHAZITBWLTIW3XSA3EXJ5IU35Q3AFGOVUSZMLQ, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.071613288 (14h15m14.586715868s) [noise=true , address=104.36.82.9:4040 , piece_id=IT2Y4ZC5F5LBQSYOHZ3YAS5GMMIR4GUFTGJQWOM2ZSNZZ7QR6LJA, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.071878812 (14h15m14.586981381s) [noise=true , address=104.36.82.10:4097 , piece_id=XL72INQPSMGOCIBWNA2GYUOKLLQKVDAQ27NX5UPZ7Y2WHXIT2SGA, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.074802672 (14h15m14.589905248s) [noise=true , address=77.90.31.17:4789 , piece_id=UAPJNIIAY4C72FKIIGLN7J4H3DHMR64YCY53T5KP2BNAODYYYNTQ, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.082112771 (14h15m14.597215343s) [noise=true , address=104.36.82.5:4850 , piece_id=RVQ3JQ3WI2QTVXI5UQHRCQPIHJB6XBE3XVKBULUEH2JEOK4FZENA, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.207237770 (14h15m14.722340347s) [noise=true , address=79.127.201.215:4351 , piece_id=VWNGLL7VJ3HPMAPSXYI6QKCNYQTPPLOJSVG7J6KOVLAKKHUCLLXQ, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.229458966 (14h15m14.744561535s) [noise=true , address=79.127.201.215:4349 , piece_id=TDGKTKU6Z4FGNXGPOPZOQXQ32YLYKICAGNB7AG26WHC6LFIWHEPA, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.259373189 (14h15m14.774475752s) [noise=true , address=104.36.82.46:4508 , piece_id=HG7TGN3Q5SETJRBS6GTVYJDN5EM7E47XBBLF57PD4TKKNCZCCKIQ, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.259396917 (14h15m14.774499494s) [seg_pos={0 0}, upload=14660]: Segment reached optimal threshold of 14 pieces. Aug 7 23:54:40.260371773 (14h15m14.77547435s) [noise=true , address=104.36.82.45:4915 , piece_id=E6ZVUDFXRP5FRVOAAQN2XF2MGVSITTCHGU5X5UVZFZZFLAUF4KNQ, seg_pos={0 0}, upload=14660]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17rmsv4edxH6jmsN83ceiVJdfEui1qrBK5XBcf7GgvKCn2fknT): protocol: read tcp 159.223.143.39:38678->104.36.82.45:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:54:40.260446597 (14h15m14.775549175s) [noise=true , address=104.36.82.58:4294 , piece_id=AWHMK4IYW5SOFHLSW5OYVOAT43EV525LGTTZA5NOVUXOHGDFGVZQ, seg_pos={0 0}, upload=14660]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12THzQNaXbHuTNCv3CAi8aG31YeFfPW5xe6JveSqKJWyGuHhffW): protocol: read tcp 159.223.143.39:43710->104.36.82.58:4294: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:54:40.260512745 (14h15m14.775615322s) [noise=true , address=104.36.82.47:4537 , piece_id=CHRH43QZTM357GGR2EWYK4P5N5POLUQQ6TA67VYEJLWQ75A6YXYQ, seg_pos={0 0}, upload=14660]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LTTCDNikuwf94KeKsQGFSu6ABXVYt1ombRsEyrZN5JC8d4Eot): protocol: read tcp 159.223.143.39:33428->104.36.82.47:4537: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:54:40.260571145 (14h15m14.775673722s) [noise=true , address=64.49.13.140:4505 , piece_id=VP5P42UVQ3VAROWEWCUDCQBSAPFI5OYX2XYAYUJWSE7WJIEQVJ6Q, seg_pos={0 0}, upload=14660]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1AbFiw33iRaTCNbSt9NmCYG1KBqys85G5VcgXndydkzgnxN4L4): protocol: read tcp 159.223.143.39:34286->64.49.13.140:4505: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:54:40.260583128 (14h15m14.775685695s) [seg_pos={0 0}, upload=14660]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:54:40.260664793 (14h15m14.775767369s) [upload=14660]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:54:40.260669477 (14h15m14.775772054s) [upload=14660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:11.229483942 (14h16m45.744586512s) [upload=14660]: Done uploading segments... Aug 7 23:56:11.229509087 (14h16m45.744611663s) [upload=14660]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:11.346464864 (14h16m45.86156744s) [upload=14687]: Starting upload Aug 7 23:56:11.346527049 (14h16m45.861629635s) [upload=14687]: Done starting upload Aug 7 23:56:11.346759296 (14h16m45.861861876s) [upload=14687]: Uploading segments... Aug 7 23:56:11.346881305 (14h16m45.861983884s) [upload=14687]: Got next segment. Inline: false Aug 7 23:56:11.346948851 (14h16m45.862051416s) [upload=14687]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:11.346955538 (14h16m45.862058115s) [upload=14687]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:57:11.314771097 (14h17m45.829873675s) [upload=14687]: Next segment err: context deadline exceeded Aug 7 23:57:11.314800306 (14h17m45.829902882s) [upload=14687]: Done uploading segments... Aug 7 23:57:11.314817175 (14h17m45.829919741s) [upload=14687]: Upload finished. err: context deadline exceeded Duration: 1m0.001408223s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 70.951352ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:57:10 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 70.975526ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 9 out of 60 is failed (threshold=3) [.XX.XXX..............................................X.XX.X.] duration: 215.333119ms FAIL (2025-08-07 23:57:09 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ed. err: <nil> Aug 7 23:49:30.817755150 (14h10m5.332857717s) [upload=14567]: Starting upload Aug 7 23:49:30.817852337 (14h10m5.332954913s) [upload=14567]: Done starting upload Aug 7 23:49:30.817934904 (14h10m5.333037472s) [upload=14567]: Uploading segments... Aug 7 23:49:30.818001589 (14h10m5.333104172s) [upload=14567]: Got next segment. Inline: true Aug 7 23:49:30.818007295 (14h10m5.333109877s) [upload=14567]: Waiting for error group managing segments... Aug 7 23:49:30.818078537 (14h10m5.333181104s) [upload=14567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:30.818084049 (14h10m5.333186629s) [upload=14567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:30.818087176 (14h10m5.333189753s) [upload=14567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:30.888782953 (14h10m5.403885532s) [upload=14567]: Done uploading segments... Aug 7 23:49:30.888817710 (14h10m5.403920277s) [upload=14567]: Upload finished. err: <nil> Aug 7 23:50:32.085473195 (14h11m6.600575774s) [upload=14591]: Starting upload Aug 7 23:50:32.085550609 (14h11m6.600653185s) [upload=14591]: Done starting upload Aug 7 23:50:32.085610615 (14h11m6.600713183s) [upload=14591]: Uploading segments... Aug 7 23:50:32.085658436 (14h11m6.600761018s) [upload=14591]: Got next segment. Inline: true Aug 7 23:50:32.085662726 (14h11m6.600765303s) [upload=14591]: Waiting for error group managing segments... Aug 7 23:50:32.085716357 (14h11m6.600818923s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:32.085721348 (14h11m6.600823924s) [upload=14591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:32.085724995 (14h11m6.600827572s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:32.142060473 (14h11m6.657163039s) [upload=14591]: Done uploading segments... Aug 7 23:50:32.142076779 (14h11m6.657179345s) [upload=14591]: Upload finished. err: <nil> Aug 7 23:50:57.356595368 (14h11m31.871697934s) [upload=14595]: Starting upload Aug 7 23:50:57.356765546 (14h11m31.871868133s) [upload=14595]: Done starting upload Aug 7 23:50:57.356869337 (14h11m31.871971899s) [upload=14595]: Uploading segments... Aug 7 23:50:57.356914930 (14h11m31.872017522s) [upload=14595]: Got next segment. Inline: true Aug 7 23:50:57.356921513 (14h11m31.872024098s) [upload=14595]: Waiting for error group managing segments... Aug 7 23:50:57.356982351 (14h11m31.872084938s) [upload=14595]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:57.356989548 (14h11m31.872092141s) [upload=14595]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:57.356995954 (14h11m31.872098543s) [upload=14595]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:04.638440451 (14h11m39.153543026s) [upload=14595]: Done uploading segments... Aug 7 23:51:04.638457479 (14h11m39.153560045s) [upload=14595]: Upload finished. err: <nil> Aug 7 23:52:06.228205136 (14h12m40.743307702s) [upload=14620]: Starting upload Aug 7 23:52:06.228273353 (14h12m40.74337593s) [upload=14620]: Done starting upload Aug 7 23:52:06.228361368 (14h12m40.743463938s) [upload=14620]: Uploading segments... Aug 7 23:52:06.228433067 (14h12m40.743535646s) [upload=14620]: Got next segment. Inline: true Aug 7 23:52:06.228437780 (14h12m40.743540356s) [upload=14620]: Waiting for error group managing segments... Aug 7 23:52:06.228479528 (14h12m40.743582104s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:06.228483802 (14h12m40.743586378s) [upload=14620]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:06.228498744 (14h12m40.743601323s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:06.292782592 (14h12m40.807885172s) [upload=14620]: Done uploading segments... Aug 7 23:52:06.292803751 (14h12m40.80790633s) [upload=14620]: Upload finished. err: <nil> Aug 7 23:52:55.204638563 (14h13m29.719741157s) [upload=14636]: Starting upload Aug 7 23:52:55.204832733 (14h13m29.719935326s) [upload=14636]: Done starting upload Aug 7 23:52:55.204913770 (14h13m29.720016348s) [upload=14636]: Uploading segments... Aug 7 23:52:55.205057568 (14h13m29.720160151s) [upload=14636]: Got next segment. Inline: true Aug 7 23:52:55.205064394 (14h13m29.720166971s) [upload=14636]: Waiting for error group managing segments... Aug 7 23:52:55.205114279 (14h13m29.720216864s) [upload=14636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:55.205134068 (14h13m29.720236652s) [upload=14636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:55.205137808 (14h13m29.72024039s) [upload=14636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:25.172932226 (14h13m59.688034802s) [upload=14636]: Done uploading segments... Aug 7 23:53:25.172962711 (14h13m59.688065287s) [upload=14636]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:25.682055912 (14h14m0.197158491s) [upload=14648]: Starting upload Aug 7 23:53:25.682122304 (14h14m0.197224872s) [upload=14648]: Done starting upload Aug 7 23:53:25.682169894 (14h14m0.197272463s) [upload=14648]: Uploading segments... Aug 7 23:53:25.682274124 (14h14m0.197376703s) [upload=14648]: Got next segment. Inline: true Aug 7 23:53:25.682279715 (14h14m0.197382292s) [upload=14648]: Waiting for error group managing segments... Aug 7 23:53:25.682336423 (14h14m0.197438989s) [upload=14648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:25.682341672 (14h14m0.197444249s) [upload=14648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:25.682344934 (14h14m0.197447509s) [upload=14648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:25.755587837 (14h14m0.270690415s) [upload=14648]: Done uploading segments... Aug 7 23:53:25.755603818 (14h14m0.270706384s) [upload=14648]: Upload finished. err: <nil> Aug 7 23:54:03.333646436 (14h14m37.848749023s) [upload=14655]: Starting upload Aug 7 23:54:03.333752884 (14h14m37.848855471s) [upload=14655]: Done starting upload Aug 7 23:54:03.333844300 (14h14m37.848946877s) [upload=14655]: Uploading segments... Aug 7 23:54:03.333909324 (14h14m37.84901191s) [upload=14655]: Got next segment. Inline: true Aug 7 23:54:03.333915381 (14h14m37.849017987s) [upload=14655]: Waiting for error group managing segments... Aug 7 23:54:03.333983662 (14h14m37.849086249s) [upload=14655]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:03.333989573 (14h14m37.849092156s) [upload=14655]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:03.333993282 (14h14m37.849095867s) [upload=14655]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:33.301833586 (14h15m7.816936169s) [upload=14655]: Done uploading segments... Aug 7 23:54:33.301863059 (14h15m7.816965638s) [upload=14655]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:36.259738412 (14h15m10.774840994s) [upload=14664]: Starting upload Aug 7 23:54:36.259845771 (14h15m10.77494834s) [upload=14664]: Done starting upload Aug 7 23:54:36.260065480 (14h15m10.775168051s) [upload=14664]: Uploading segments... Aug 7 23:54:36.260143124 (14h15m10.775245714s) [upload=14664]: Got next segment. Inline: true Aug 7 23:54:36.260148821 (14h15m10.775251404s) [upload=14664]: Waiting for error group managing segments... Aug 7 23:54:36.260217622 (14h15m10.775320206s) [upload=14664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:36.260223500 (14h15m10.77532608s) [upload=14664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:36.260227779 (14h15m10.77533036s) [upload=14664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:06.222352771 (14h15m40.737455348s) [upload=14664]: Done uploading segments... Aug 7 23:55:06.222372750 (14h15m40.737475327s) [upload=14664]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:36.453323212 (14h16m10.968425795s) [upload=14680]: Starting upload Aug 7 23:55:36.453397671 (14h16m10.968500249s) [upload=14680]: Done starting upload Aug 7 23:55:36.453487058 (14h16m10.968589634s) [upload=14680]: Uploading segments... Aug 7 23:55:36.453541640 (14h16m10.968644227s) [upload=14680]: Got next segment. Inline: true Aug 7 23:55:36.453546785 (14h16m10.968649366s) [upload=14680]: Waiting for error group managing segments... Aug 7 23:55:36.453592204 (14h16m10.968694785s) [upload=14680]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:36.453604678 (14h16m10.968707261s) [upload=14680]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:36.453608102 (14h16m10.96871068s) [upload=14680]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:36.518300419 (14h16m11.033402995s) [upload=14680]: Done uploading segments... Aug 7 23:55:36.518315426 (14h16m11.033417992s) [upload=14680]: Upload finished. err: <nil> Aug 7 23:56:22.913855555 (14h16m57.428958117s) [upload=14689]: Starting upload Aug 7 23:56:22.913925154 (14h16m57.429027734s) [upload=14689]: Done starting upload Aug 7 23:56:22.913999531 (14h16m57.429102093s) [upload=14689]: Uploading segments... Aug 7 23:56:22.914048699 (14h16m57.429151282s) [upload=14689]: Got next segment. Inline: true Aug 7 23:56:22.914052623 (14h16m57.4291552s) [upload=14689]: Waiting for error group managing segments... Aug 7 23:56:22.914089928 (14h16m57.429192503s) [upload=14689]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:22.914093434 (14h16m57.42919601s) [upload=14689]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:22.914096175 (14h16m57.429198751s) [upload=14689]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:52.884393262 (14h17m27.399495844s) [upload=14689]: Done uploading segments... Aug 7 23:56:52.884415558 (14h17m27.399518126s) [upload=14689]: Upload finished. err: metaclient: context deadline exceeded Duration: 215.380762ms core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 44.89148ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 23:57:03 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 44.915675ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 11 out of 60 is failed (threshold=3) [..XXXXX.X.XX...........................................X..XX] duration: 30.001767875s FAIL (2025-08-07 23:56:56 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001799225s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........X...........................................XXX.X..] duration: 369.285237ms FAIL (2025-08-07 23:56:52 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== (14h15m11.016995473s) [upload=14665]: Uploading segments... Aug 7 23:54:36.502140541 (14h15m11.017243126s) [upload=14665]: Got next segment. Inline: false Aug 7 23:54:36.502208316 (14h15m11.017310893s) [upload=14665]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:36.502213415 (14h15m11.017315993s) [upload=14665]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:54:36.540470046 (14h15m11.055572633s) [seg_pos={0 0}, upload=14665]: Begin upload segment... Aug 7 23:54:36.540542869 (14h15m11.055645435s) [seg_pos={0 0}, upload=14665]: Done begin upload segment. Aug 7 23:54:36.540586398 (14h15m11.055688977s) [noise=true , address=77.90.31.10:4520 , piece_id=UXBIKFQ4KOGFTCR73EVF7DBFLAZXOJPPHZUVL4BTJIONLHHQOF3Q, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.540599057 (14h15m11.055701646s) [noise=true , address=104.36.82.7:4909 , piece_id=ISAGUZRBSZZOI5D3XK4N2VBH6CGHWALHK7DUKKDKYTA3PH54WE7A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.541179792 (14h15m11.056282359s) [noise=true , address=45.91.116.6:4975 , piece_id=GSG54APU3E2NKOOHZDOELFVKDEWZPDNYAC6GUW63N2WHS54AHJ5A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.541219031 (14h15m11.056321617s) [noise=true , address=45.91.116.7:4010 , piece_id=HIHBEZ2SFGNUPDJSUPBGYJYIQE5LT5U3KIOWJWTKQWNWHSMB2R6Q, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.541591033 (14h15m11.056693604s) [noise=true , address=104.36.82.5:4837 , piece_id=FJIF5DAJKKHVPDI4FJQFRIY66V6LQXEN7GVMZLCWFPG35QJGSO3Q, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.541808787 (14h15m11.056911375s) [noise=true , address=104.36.82.13:4185 , piece_id=DY4ZRJG7XBEMN5JTETQR5I6HSLLQNIT3R6ZS7CA5ZBW6QAQVGXSA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.542414303 (14h15m11.057516892s) [noise=true , address=104.36.82.16:4294 , piece_id=UHSQZJSGHU3UWAEEOTG5P4NV2YGTGKYYJU7CQ7UC4J6WDUJPUGJA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.542588591 (14h15m11.057691157s) [noise=true , address=104.36.82.4:4802 , piece_id=QHJY5W7S3OB2TAIBARTKQTB6HHBRGKPZP2CM7F4V7NYZQLZZSQ7A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.542987544 (14h15m11.058090129s) [noise=true , address=104.36.82.8:4972 , piece_id=F3E53EQXBSUIZDZV5L5CBCYYVE5BOEMSTVEZZMEI7NKKHHKPD2LQ, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.543563707 (14h15m11.058666294s) [noise=true , address=104.36.82.8:4971 , piece_id=HC2FEV5IWEXHECRHUAMAF7J7J4AWYZRTLRD5NUREB7SBCUEA4VGQ, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.543788500 (14h15m11.058891086s) [noise=true , address=104.36.82.6:4876 , piece_id=BABG2QF67NQHDH62KKGR2FLZRTF5TWGBNQ5D2PKQL6QJURYIROGA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.544354038 (14h15m11.059456606s) [noise=true , address=77.90.31.12:4589 , piece_id=L4B3OU3E72GCPP4SZLWPJ2B3LNS6GSFX3FFWQBCVBHL3JGHR7MGA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.544746164 (14h15m11.059848744s) [noise=true , address=104.36.82.2:4730 , piece_id=KY5B7UDNU2S2ZFHCVWZAPJV4QHEMCT4RKKJLOM3SXKTC22HVGKVA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.545099366 (14h15m11.060201945s) [noise=true , address=77.90.31.19:4841 , piece_id=QV43X4G6B3IXCVDVAJYSGXUUI6LCROUL3IHGNES5JTUEPGKNFA5A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.545536838 (14h15m11.060639427s) [noise=true , address=77.90.31.14:4663 , piece_id=6OHGW5KCETKDQUXWUUD7RSP6JXHCKTSCPAA22E3QP46OWHE4A2FQ, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.546110762 (14h15m11.061213352s) [noise=true , address=77.90.31.17:4771 , piece_id=EH7SHR3OQ3ODCJ2QKQBVROK3BNG6E4B6PKMTNQMDUA4RRZAGAZ2A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.648423840 (14h15m11.16352641s) [upload=14665]: Next returned nil segment Aug 7 23:54:36.648440535 (14h15m11.163543117s) [upload=14665]: Waiting for error group managing segments... Aug 7 23:54:36.654558819 (14h15m11.169661399s) [noise=true , address=45.91.116.7:4010 , piece_id=HIHBEZ2SFGNUPDJSUPBGYJYIQE5LT5U3KIOWJWTKQWNWHSMB2R6Q, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.655196697 (14h15m11.170299261s) [noise=true , address=77.90.31.10:4520 , piece_id=UXBIKFQ4KOGFTCR73EVF7DBFLAZXOJPPHZUVL4BTJIONLHHQOF3Q, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.658908365 (14h15m11.174010932s) [noise=true , address=77.90.31.17:4771 , piece_id=EH7SHR3OQ3ODCJ2QKQBVROK3BNG6E4B6PKMTNQMDUA4RRZAGAZ2A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.659089278 (14h15m11.174191848s) [noise=true , address=45.91.116.6:4975 , piece_id=GSG54APU3E2NKOOHZDOELFVKDEWZPDNYAC6GUW63N2WHS54AHJ5A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.660774579 (14h15m11.175877148s) [noise=true , address=77.90.31.14:4663 , piece_id=6OHGW5KCETKDQUXWUUD7RSP6JXHCKTSCPAA22E3QP46OWHE4A2FQ, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.661054908 (14h15m11.176157476s) [noise=true , address=77.90.31.12:4589 , piece_id=L4B3OU3E72GCPP4SZLWPJ2B3LNS6GSFX3FFWQBCVBHL3JGHR7MGA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.662842221 (14h15m11.17794479s) [noise=true , address=77.90.31.19:4841 , piece_id=QV43X4G6B3IXCVDVAJYSGXUUI6LCROUL3IHGNES5JTUEPGKNFA5A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.663362200 (14h15m11.178464776s) [noise=true , address=104.36.82.6:4876 , piece_id=BABG2QF67NQHDH62KKGR2FLZRTF5TWGBNQ5D2PKQL6QJURYIROGA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.666343769 (14h15m11.181446338s) [noise=true , address=104.36.82.16:4294 , piece_id=UHSQZJSGHU3UWAEEOTG5P4NV2YGTGKYYJU7CQ7UC4J6WDUJPUGJA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.675326798 (14h15m11.190429377s) [noise=true , address=104.36.82.2:4730 , piece_id=KY5B7UDNU2S2ZFHCVWZAPJV4QHEMCT4RKKJLOM3SXKTC22HVGKVA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.678428557 (14h15m11.193531125s) [noise=true , address=104.36.82.4:4802 , piece_id=QHJY5W7S3OB2TAIBARTKQTB6HHBRGKPZP2CM7F4V7NYZQLZZSQ7A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.685873433 (14h15m11.200976001s) [noise=true , address=104.36.82.8:4972 , piece_id=F3E53EQXBSUIZDZV5L5CBCYYVE5BOEMSTVEZZMEI7NKKHHKPD2LQ, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.693259479 (14h15m11.208362058s) [noise=true , address=104.36.82.8:4971 , piece_id=HC2FEV5IWEXHECRHUAMAF7J7J4AWYZRTLRD5NUREB7SBCUEA4VGQ, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.693284102 (14h15m11.208386678s) [seg_pos={0 0}, upload=14665]: Segment reached optimal threshold of 13 pieces. Aug 7 23:54:36.693493083 (14h15m11.20859566s) [noise=true , address=104.36.82.7:4909 , piece_id=ISAGUZRBSZZOI5D3XK4N2VBH6CGHWALHK7DUKKDKYTA3PH54WE7A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF Aug 7 23:54:36.693795135 (14h15m11.208897711s) [noise=true , address=104.36.82.5:4837 , piece_id=FJIF5DAJKKHVPDI4FJQFRIY66V6LQXEN7GVMZLCWFPG35QJGSO3Q, seg_pos={0 0}, upload=14665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:55994->104.36.82.5:4837: use of closed network connection; write tcp 159.223.143.39:55994->104.36.82.5:4837: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:55994->104.36.82.5:4837: use of closed network connection Aug 7 23:54:36.693865018 (14h15m11.208967588s) [noise=true , address=104.36.82.13:4185 , piece_id=DY4ZRJG7XBEMN5JTETQR5I6HSLLQNIT3R6ZS7CA5ZBW6QAQVGXSA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: read tcp 159.223.143.39:52730->104.36.82.13:4185: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:54:36.693875333 (14h15m11.208977898s) [seg_pos={0 0}, upload=14665]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:54:36.693954210 (14h15m11.209056786s) [upload=14665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:54:36.693959007 (14h15m11.209061583s) [upload=14665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:36.756106678 (14h15m11.271209244s) [upload=14665]: Done uploading segments... Aug 7 23:54:36.756123317 (14h15m11.271225887s) [upload=14665]: Upload finished. err: <nil> Aug 7 23:55:38.250097734 (14h16m12.765200314s) [upload=14681]: Starting upload Aug 7 23:55:38.250192737 (14h16m12.765295302s) [upload=14681]: Done starting upload Aug 7 23:55:38.250341846 (14h16m12.765444411s) [upload=14681]: Uploading segments... Aug 7 23:55:38.250743926 (14h16m12.76584651s) [upload=14681]: Got next segment. Inline: false Aug 7 23:55:38.250809082 (14h16m12.765911663s) [upload=14681]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:38.250814071 (14h16m12.765916648s) [upload=14681]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:56:08.219284519 (14h16m42.734387085s) [upload=14681]: Next segment err: context deadline exceeded Aug 7 23:56:08.219364566 (14h16m42.734467135s) [upload=14681]: Done uploading segments... Aug 7 23:56:08.219382617 (14h16m42.734485184s) [upload=14681]: Upload finished. err: context deadline exceeded Duration: 369.315836ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 9 out of 60 is failed (threshold=3) [..XX.XXX..............................................X.XX.X] duration: 30.001381845s FAIL (2025-08-07 23:56:52 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ed. err: <nil> Aug 7 23:49:30.817755150 (14h10m5.332857717s) [upload=14567]: Starting upload Aug 7 23:49:30.817852337 (14h10m5.332954913s) [upload=14567]: Done starting upload Aug 7 23:49:30.817934904 (14h10m5.333037472s) [upload=14567]: Uploading segments... Aug 7 23:49:30.818001589 (14h10m5.333104172s) [upload=14567]: Got next segment. Inline: true Aug 7 23:49:30.818007295 (14h10m5.333109877s) [upload=14567]: Waiting for error group managing segments... Aug 7 23:49:30.818078537 (14h10m5.333181104s) [upload=14567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:30.818084049 (14h10m5.333186629s) [upload=14567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:30.818087176 (14h10m5.333189753s) [upload=14567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:30.888782953 (14h10m5.403885532s) [upload=14567]: Done uploading segments... Aug 7 23:49:30.888817710 (14h10m5.403920277s) [upload=14567]: Upload finished. err: <nil> Aug 7 23:50:32.085473195 (14h11m6.600575774s) [upload=14591]: Starting upload Aug 7 23:50:32.085550609 (14h11m6.600653185s) [upload=14591]: Done starting upload Aug 7 23:50:32.085610615 (14h11m6.600713183s) [upload=14591]: Uploading segments... Aug 7 23:50:32.085658436 (14h11m6.600761018s) [upload=14591]: Got next segment. Inline: true Aug 7 23:50:32.085662726 (14h11m6.600765303s) [upload=14591]: Waiting for error group managing segments... Aug 7 23:50:32.085716357 (14h11m6.600818923s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:32.085721348 (14h11m6.600823924s) [upload=14591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:32.085724995 (14h11m6.600827572s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:32.142060473 (14h11m6.657163039s) [upload=14591]: Done uploading segments... Aug 7 23:50:32.142076779 (14h11m6.657179345s) [upload=14591]: Upload finished. err: <nil> Aug 7 23:50:57.356595368 (14h11m31.871697934s) [upload=14595]: Starting upload Aug 7 23:50:57.356765546 (14h11m31.871868133s) [upload=14595]: Done starting upload Aug 7 23:50:57.356869337 (14h11m31.871971899s) [upload=14595]: Uploading segments... Aug 7 23:50:57.356914930 (14h11m31.872017522s) [upload=14595]: Got next segment. Inline: true Aug 7 23:50:57.356921513 (14h11m31.872024098s) [upload=14595]: Waiting for error group managing segments... Aug 7 23:50:57.356982351 (14h11m31.872084938s) [upload=14595]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:57.356989548 (14h11m31.872092141s) [upload=14595]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:57.356995954 (14h11m31.872098543s) [upload=14595]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:04.638440451 (14h11m39.153543026s) [upload=14595]: Done uploading segments... Aug 7 23:51:04.638457479 (14h11m39.153560045s) [upload=14595]: Upload finished. err: <nil> Aug 7 23:52:06.228205136 (14h12m40.743307702s) [upload=14620]: Starting upload Aug 7 23:52:06.228273353 (14h12m40.74337593s) [upload=14620]: Done starting upload Aug 7 23:52:06.228361368 (14h12m40.743463938s) [upload=14620]: Uploading segments... Aug 7 23:52:06.228433067 (14h12m40.743535646s) [upload=14620]: Got next segment. Inline: true Aug 7 23:52:06.228437780 (14h12m40.743540356s) [upload=14620]: Waiting for error group managing segments... Aug 7 23:52:06.228479528 (14h12m40.743582104s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:06.228483802 (14h12m40.743586378s) [upload=14620]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:06.228498744 (14h12m40.743601323s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:06.292782592 (14h12m40.807885172s) [upload=14620]: Done uploading segments... Aug 7 23:52:06.292803751 (14h12m40.80790633s) [upload=14620]: Upload finished. err: <nil> Aug 7 23:52:55.204638563 (14h13m29.719741157s) [upload=14636]: Starting upload Aug 7 23:52:55.204832733 (14h13m29.719935326s) [upload=14636]: Done starting upload Aug 7 23:52:55.204913770 (14h13m29.720016348s) [upload=14636]: Uploading segments... Aug 7 23:52:55.205057568 (14h13m29.720160151s) [upload=14636]: Got next segment. Inline: true Aug 7 23:52:55.205064394 (14h13m29.720166971s) [upload=14636]: Waiting for error group managing segments... Aug 7 23:52:55.205114279 (14h13m29.720216864s) [upload=14636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:55.205134068 (14h13m29.720236652s) [upload=14636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:55.205137808 (14h13m29.72024039s) [upload=14636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:25.172932226 (14h13m59.688034802s) [upload=14636]: Done uploading segments... Aug 7 23:53:25.172962711 (14h13m59.688065287s) [upload=14636]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:25.682055912 (14h14m0.197158491s) [upload=14648]: Starting upload Aug 7 23:53:25.682122304 (14h14m0.197224872s) [upload=14648]: Done starting upload Aug 7 23:53:25.682169894 (14h14m0.197272463s) [upload=14648]: Uploading segments... Aug 7 23:53:25.682274124 (14h14m0.197376703s) [upload=14648]: Got next segment. Inline: true Aug 7 23:53:25.682279715 (14h14m0.197382292s) [upload=14648]: Waiting for error group managing segments... Aug 7 23:53:25.682336423 (14h14m0.197438989s) [upload=14648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:25.682341672 (14h14m0.197444249s) [upload=14648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:25.682344934 (14h14m0.197447509s) [upload=14648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:25.755587837 (14h14m0.270690415s) [upload=14648]: Done uploading segments... Aug 7 23:53:25.755603818 (14h14m0.270706384s) [upload=14648]: Upload finished. err: <nil> Aug 7 23:54:03.333646436 (14h14m37.848749023s) [upload=14655]: Starting upload Aug 7 23:54:03.333752884 (14h14m37.848855471s) [upload=14655]: Done starting upload Aug 7 23:54:03.333844300 (14h14m37.848946877s) [upload=14655]: Uploading segments... Aug 7 23:54:03.333909324 (14h14m37.84901191s) [upload=14655]: Got next segment. Inline: true Aug 7 23:54:03.333915381 (14h14m37.849017987s) [upload=14655]: Waiting for error group managing segments... Aug 7 23:54:03.333983662 (14h14m37.849086249s) [upload=14655]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:03.333989573 (14h14m37.849092156s) [upload=14655]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:03.333993282 (14h14m37.849095867s) [upload=14655]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:33.301833586 (14h15m7.816936169s) [upload=14655]: Done uploading segments... Aug 7 23:54:33.301863059 (14h15m7.816965638s) [upload=14655]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:36.259738412 (14h15m10.774840994s) [upload=14664]: Starting upload Aug 7 23:54:36.259845771 (14h15m10.77494834s) [upload=14664]: Done starting upload Aug 7 23:54:36.260065480 (14h15m10.775168051s) [upload=14664]: Uploading segments... Aug 7 23:54:36.260143124 (14h15m10.775245714s) [upload=14664]: Got next segment. Inline: true Aug 7 23:54:36.260148821 (14h15m10.775251404s) [upload=14664]: Waiting for error group managing segments... Aug 7 23:54:36.260217622 (14h15m10.775320206s) [upload=14664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:36.260223500 (14h15m10.77532608s) [upload=14664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:36.260227779 (14h15m10.77533036s) [upload=14664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:06.222352771 (14h15m40.737455348s) [upload=14664]: Done uploading segments... Aug 7 23:55:06.222372750 (14h15m40.737475327s) [upload=14664]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:36.453323212 (14h16m10.968425795s) [upload=14680]: Starting upload Aug 7 23:55:36.453397671 (14h16m10.968500249s) [upload=14680]: Done starting upload Aug 7 23:55:36.453487058 (14h16m10.968589634s) [upload=14680]: Uploading segments... Aug 7 23:55:36.453541640 (14h16m10.968644227s) [upload=14680]: Got next segment. Inline: true Aug 7 23:55:36.453546785 (14h16m10.968649366s) [upload=14680]: Waiting for error group managing segments... Aug 7 23:55:36.453592204 (14h16m10.968694785s) [upload=14680]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:36.453604678 (14h16m10.968707261s) [upload=14680]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:36.453608102 (14h16m10.96871068s) [upload=14680]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:36.518300419 (14h16m11.033402995s) [upload=14680]: Done uploading segments... Aug 7 23:55:36.518315426 (14h16m11.033417992s) [upload=14680]: Upload finished. err: <nil> Aug 7 23:56:22.913855555 (14h16m57.428958117s) [upload=14689]: Starting upload Aug 7 23:56:22.913925154 (14h16m57.429027734s) [upload=14689]: Done starting upload Aug 7 23:56:22.913999531 (14h16m57.429102093s) [upload=14689]: Uploading segments... Aug 7 23:56:22.914048699 (14h16m57.429151282s) [upload=14689]: Got next segment. Inline: true Aug 7 23:56:22.914052623 (14h16m57.4291552s) [upload=14689]: Waiting for error group managing segments... Aug 7 23:56:22.914089928 (14h16m57.429192503s) [upload=14689]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:22.914093434 (14h16m57.42919601s) [upload=14689]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:22.914096175 (14h16m57.429198751s) [upload=14689]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:52.884393262 (14h17m27.399495844s) [upload=14689]: Done uploading segments... Aug 7 23:56:52.884415558 (14h17m27.399518126s) [upload=14689]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.001403346s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...XXX..X.X.X.....................................X.........] duration: 543.11767ms FAIL (2025-08-07 23:56:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 543.381468ms edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 39.773787ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:56:42 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 39.821966ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [X.X.X....XX..........................................X..X...] duration: 263.135742ms FAIL (2025-08-07 23:56:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 263.159119ms edge gateway us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 29.773493ms target: https://us1.storj.io/login FAIL (2025-08-07 23:56:37 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 29.797226ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [..XX..X...X..............................................XXX] duration: 1m0.001335867s FAIL (2025-08-07 23:56:22 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 84613s) [upload=14523]: Upload finished. err: <nil> Aug 7 23:47:43.879457284 (14h8m18.394559871s) [upload=14540]: Starting upload Aug 7 23:47:43.879610635 (14h8m18.394713221s) [upload=14540]: Done starting upload Aug 7 23:47:43.879713914 (14h8m18.394816491s) [upload=14540]: Uploading segments... Aug 7 23:47:43.879808496 (14h8m18.39491109s) [upload=14540]: Got next segment. Inline: true Aug 7 23:47:43.879816280 (14h8m18.394918867s) [upload=14540]: Waiting for error group managing segments... Aug 7 23:47:43.879892463 (14h8m18.394995049s) [upload=14540]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:43.879898862 (14h8m18.395001442s) [upload=14540]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:43.879902786 (14h8m18.395005367s) [upload=14540]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:43.948673039 (14h8m18.463775606s) [upload=14540]: Done uploading segments... Aug 7 23:47:43.948690272 (14h8m18.463792838s) [upload=14540]: Upload finished. err: <nil> Aug 7 23:48:32.733662762 (14h9m7.248765339s) [upload=14555]: Starting upload Aug 7 23:48:32.733764614 (14h9m7.248867195s) [upload=14555]: Done starting upload Aug 7 23:48:32.733839718 (14h9m7.248942295s) [upload=14555]: Uploading segments... Aug 7 23:48:32.733888071 (14h9m7.248990659s) [upload=14555]: Got next segment. Inline: true Aug 7 23:48:32.733893767 (14h9m7.248996349s) [upload=14555]: Waiting for error group managing segments... Aug 7 23:48:32.733943638 (14h9m7.249046215s) [upload=14555]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:32.733949815 (14h9m7.249052399s) [upload=14555]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:32.733954110 (14h9m7.24905669s) [upload=14555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:32.809665685 (14h9m7.324768252s) [upload=14555]: Done uploading segments... Aug 7 23:48:32.809699832 (14h9m7.324802408s) [upload=14555]: Upload finished. err: <nil> Aug 7 23:49:43.381878590 (14h10m17.896981166s) [upload=14573]: Starting upload Aug 7 23:49:43.382020923 (14h10m17.897123495s) [upload=14573]: Done starting upload Aug 7 23:49:43.382128913 (14h10m17.897231484s) [upload=14573]: Uploading segments... Aug 7 23:49:43.382204063 (14h10m17.897306641s) [upload=14573]: Got next segment. Inline: true Aug 7 23:49:43.382211147 (14h10m17.897313718s) [upload=14573]: Waiting for error group managing segments... Aug 7 23:49:43.382287220 (14h10m17.897389786s) [upload=14573]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:43.382292849 (14h10m17.897395421s) [upload=14573]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:43.382297255 (14h10m17.897399833s) [upload=14573]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:43.532367962 (14h10m18.047470542s) [upload=14573]: Done uploading segments... Aug 7 23:49:43.532399853 (14h10m18.047502429s) [upload=14573]: Upload finished. err: <nil> Aug 7 23:50:25.794985700 (14h11m0.310088267s) [upload=14588]: Starting upload Aug 7 23:50:25.795091135 (14h11m0.310193704s) [upload=14588]: Done starting upload Aug 7 23:50:25.795192505 (14h11m0.310295073s) [upload=14588]: Uploading segments... Aug 7 23:50:25.795250709 (14h11m0.310353286s) [upload=14588]: Got next segment. Inline: true Aug 7 23:50:25.795256599 (14h11m0.310359177s) [upload=14588]: Waiting for error group managing segments... Aug 7 23:50:25.795298950 (14h11m0.310401527s) [upload=14588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:25.795303481 (14h11m0.310406058s) [upload=14588]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:25.795306901 (14h11m0.310409476s) [upload=14588]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:25.886991014 (14h11m0.402093596s) [upload=14588]: Done uploading segments... Aug 7 23:50:25.887011651 (14h11m0.402114228s) [upload=14588]: Upload finished. err: <nil> Aug 7 23:51:02.166684989 (14h11m36.681787571s) [upload=14598]: Starting upload Aug 7 23:51:02.166793315 (14h11m36.681895904s) [upload=14598]: Done starting upload Aug 7 23:51:02.167617113 (14h11m36.682719692s) [upload=14598]: Uploading segments... Aug 7 23:51:02.167688699 (14h11m36.682791288s) [upload=14598]: Got next segment. Inline: true Aug 7 23:51:02.167696965 (14h11m36.68279955s) [upload=14598]: Waiting for error group managing segments... Aug 7 23:51:02.167768986 (14h11m36.682871569s) [upload=14598]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:02.167776149 (14h11m36.682878967s) [upload=14598]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:02.167781308 (14h11m36.682883888s) [upload=14598]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:02.249738740 (14h11m36.764841311s) [upload=14598]: Done uploading segments... Aug 7 23:51:02.249754857 (14h11m36.764857435s) [upload=14598]: Upload finished. err: <nil> Aug 7 23:51:48.536453050 (14h12m23.051555623s) [upload=14610]: Starting upload Aug 7 23:51:48.536534615 (14h12m23.051637193s) [upload=14610]: Done starting upload Aug 7 23:51:48.536600634 (14h12m23.051703211s) [upload=14610]: Uploading segments... Aug 7 23:51:48.536652707 (14h12m23.051755293s) [upload=14610]: Got next segment. Inline: true Aug 7 23:51:48.536658284 (14h12m23.051760862s) [upload=14610]: Waiting for error group managing segments... Aug 7 23:51:48.536705902 (14h12m23.051808486s) [upload=14610]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:48.536710850 (14h12m23.051813429s) [upload=14610]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:48.536714377 (14h12m23.051816954s) [upload=14610]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:48.601979182 (14h12m23.117081769s) [upload=14610]: Done uploading segments... Aug 7 23:51:48.602001648 (14h12m23.117104229s) [upload=14610]: Upload finished. err: <nil> Aug 7 23:52:04.519337006 (14h12m39.034439591s) [upload=14619]: Starting upload Aug 7 23:52:04.519460694 (14h12m39.034563258s) [upload=14619]: Done starting upload Aug 7 23:52:04.519569621 (14h12m39.034672186s) [upload=14619]: Uploading segments... Aug 7 23:52:04.519675504 (14h12m39.034778091s) [upload=14619]: Got next segment. Inline: true Aug 7 23:52:04.519701072 (14h12m39.034803649s) [upload=14619]: Waiting for error group managing segments... Aug 7 23:52:04.519755555 (14h12m39.034858131s) [upload=14619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:04.519760650 (14h12m39.034863226s) [upload=14619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:04.519777078 (14h12m39.034879654s) [upload=14619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:04.588422656 (14h12m39.103525241s) [upload=14619]: Done uploading segments... Aug 7 23:52:04.588444719 (14h12m39.103547297s) [upload=14619]: Upload finished. err: <nil> Aug 7 23:53:08.147087231 (14h13m42.662189804s) [upload=14641]: Starting upload Aug 7 23:53:08.147181411 (14h13m42.662283996s) [upload=14641]: Done starting upload Aug 7 23:53:08.147281202 (14h13m42.66238377s) [upload=14641]: Uploading segments... Aug 7 23:53:08.147322830 (14h13m42.662425415s) [upload=14641]: Got next segment. Inline: true Aug 7 23:53:08.147327701 (14h13m42.662430277s) [upload=14641]: Waiting for error group managing segments... Aug 7 23:53:08.147455761 (14h13m42.66255833s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:08.147460709 (14h13m42.662563285s) [upload=14641]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:08.147463890 (14h13m42.662566457s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:08.116969456 (14h14m42.632072039s) [upload=14641]: Done uploading segments... Aug 7 23:54:08.117007036 (14h14m42.632109618s) [upload=14641]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:22.831162336 (14h14m57.346264913s) [upload=14661]: Starting upload Aug 7 23:54:22.831234431 (14h14m57.346337008s) [upload=14661]: Done starting upload Aug 7 23:54:22.831306794 (14h14m57.346409362s) [upload=14661]: Uploading segments... Aug 7 23:54:22.831343257 (14h14m57.346445842s) [upload=14661]: Got next segment. Inline: true Aug 7 23:54:22.831347808 (14h14m57.346450375s) [upload=14661]: Waiting for error group managing segments... Aug 7 23:54:22.831399492 (14h14m57.346502071s) [upload=14661]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:22.831404349 (14h14m57.346506915s) [upload=14661]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:22.831407696 (14h14m57.346510265s) [upload=14661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:22.799123829 (14h15m57.314226409s) [upload=14661]: Done uploading segments... Aug 7 23:55:22.799154191 (14h15m57.314256772s) [upload=14661]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:22.918485444 (14h15m57.433588022s) [upload=14677]: Starting upload Aug 7 23:55:22.918566621 (14h15m57.433672141s) [upload=14677]: Done starting upload Aug 7 23:55:22.918647605 (14h15m57.433750172s) [upload=14677]: Uploading segments... Aug 7 23:55:22.918705315 (14h15m57.4338079s) [upload=14677]: Got next segment. Inline: true Aug 7 23:55:22.918710403 (14h15m57.43381298s) [upload=14677]: Waiting for error group managing segments... Aug 7 23:55:22.918764176 (14h15m57.433866753s) [upload=14677]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:22.918769239 (14h15m57.433871815s) [upload=14677]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:22.918772352 (14h15m57.433874928s) [upload=14677]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:22.891103920 (14h16m57.406206502s) [upload=14677]: Done uploading segments... Aug 7 23:56:22.891142892 (14h16m57.406245471s) [upload=14677]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.001352021s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 31.976997ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 23:56:20 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 32.003148ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........X...........................................XXX.X.] duration: 407.322165ms FAIL (2025-08-07 23:56:13 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== (14h15m11.016995473s) [upload=14665]: Uploading segments... Aug 7 23:54:36.502140541 (14h15m11.017243126s) [upload=14665]: Got next segment. Inline: false Aug 7 23:54:36.502208316 (14h15m11.017310893s) [upload=14665]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:36.502213415 (14h15m11.017315993s) [upload=14665]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:54:36.540470046 (14h15m11.055572633s) [seg_pos={0 0}, upload=14665]: Begin upload segment... Aug 7 23:54:36.540542869 (14h15m11.055645435s) [seg_pos={0 0}, upload=14665]: Done begin upload segment. Aug 7 23:54:36.540586398 (14h15m11.055688977s) [noise=true , address=77.90.31.10:4520 , piece_id=UXBIKFQ4KOGFTCR73EVF7DBFLAZXOJPPHZUVL4BTJIONLHHQOF3Q, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.540599057 (14h15m11.055701646s) [noise=true , address=104.36.82.7:4909 , piece_id=ISAGUZRBSZZOI5D3XK4N2VBH6CGHWALHK7DUKKDKYTA3PH54WE7A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.541179792 (14h15m11.056282359s) [noise=true , address=45.91.116.6:4975 , piece_id=GSG54APU3E2NKOOHZDOELFVKDEWZPDNYAC6GUW63N2WHS54AHJ5A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.541219031 (14h15m11.056321617s) [noise=true , address=45.91.116.7:4010 , piece_id=HIHBEZ2SFGNUPDJSUPBGYJYIQE5LT5U3KIOWJWTKQWNWHSMB2R6Q, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.541591033 (14h15m11.056693604s) [noise=true , address=104.36.82.5:4837 , piece_id=FJIF5DAJKKHVPDI4FJQFRIY66V6LQXEN7GVMZLCWFPG35QJGSO3Q, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.541808787 (14h15m11.056911375s) [noise=true , address=104.36.82.13:4185 , piece_id=DY4ZRJG7XBEMN5JTETQR5I6HSLLQNIT3R6ZS7CA5ZBW6QAQVGXSA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.542414303 (14h15m11.057516892s) [noise=true , address=104.36.82.16:4294 , piece_id=UHSQZJSGHU3UWAEEOTG5P4NV2YGTGKYYJU7CQ7UC4J6WDUJPUGJA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.542588591 (14h15m11.057691157s) [noise=true , address=104.36.82.4:4802 , piece_id=QHJY5W7S3OB2TAIBARTKQTB6HHBRGKPZP2CM7F4V7NYZQLZZSQ7A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.542987544 (14h15m11.058090129s) [noise=true , address=104.36.82.8:4972 , piece_id=F3E53EQXBSUIZDZV5L5CBCYYVE5BOEMSTVEZZMEI7NKKHHKPD2LQ, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.543563707 (14h15m11.058666294s) [noise=true , address=104.36.82.8:4971 , piece_id=HC2FEV5IWEXHECRHUAMAF7J7J4AWYZRTLRD5NUREB7SBCUEA4VGQ, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.543788500 (14h15m11.058891086s) [noise=true , address=104.36.82.6:4876 , piece_id=BABG2QF67NQHDH62KKGR2FLZRTF5TWGBNQ5D2PKQL6QJURYIROGA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.544354038 (14h15m11.059456606s) [noise=true , address=77.90.31.12:4589 , piece_id=L4B3OU3E72GCPP4SZLWPJ2B3LNS6GSFX3FFWQBCVBHL3JGHR7MGA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.544746164 (14h15m11.059848744s) [noise=true , address=104.36.82.2:4730 , piece_id=KY5B7UDNU2S2ZFHCVWZAPJV4QHEMCT4RKKJLOM3SXKTC22HVGKVA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.545099366 (14h15m11.060201945s) [noise=true , address=77.90.31.19:4841 , piece_id=QV43X4G6B3IXCVDVAJYSGXUUI6LCROUL3IHGNES5JTUEPGKNFA5A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.545536838 (14h15m11.060639427s) [noise=true , address=77.90.31.14:4663 , piece_id=6OHGW5KCETKDQUXWUUD7RSP6JXHCKTSCPAA22E3QP46OWHE4A2FQ, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.546110762 (14h15m11.061213352s) [noise=true , address=77.90.31.17:4771 , piece_id=EH7SHR3OQ3ODCJ2QKQBVROK3BNG6E4B6PKMTNQMDUA4RRZAGAZ2A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.648423840 (14h15m11.16352641s) [upload=14665]: Next returned nil segment Aug 7 23:54:36.648440535 (14h15m11.163543117s) [upload=14665]: Waiting for error group managing segments... Aug 7 23:54:36.654558819 (14h15m11.169661399s) [noise=true , address=45.91.116.7:4010 , piece_id=HIHBEZ2SFGNUPDJSUPBGYJYIQE5LT5U3KIOWJWTKQWNWHSMB2R6Q, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.655196697 (14h15m11.170299261s) [noise=true , address=77.90.31.10:4520 , piece_id=UXBIKFQ4KOGFTCR73EVF7DBFLAZXOJPPHZUVL4BTJIONLHHQOF3Q, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.658908365 (14h15m11.174010932s) [noise=true , address=77.90.31.17:4771 , piece_id=EH7SHR3OQ3ODCJ2QKQBVROK3BNG6E4B6PKMTNQMDUA4RRZAGAZ2A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.659089278 (14h15m11.174191848s) [noise=true , address=45.91.116.6:4975 , piece_id=GSG54APU3E2NKOOHZDOELFVKDEWZPDNYAC6GUW63N2WHS54AHJ5A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.660774579 (14h15m11.175877148s) [noise=true , address=77.90.31.14:4663 , piece_id=6OHGW5KCETKDQUXWUUD7RSP6JXHCKTSCPAA22E3QP46OWHE4A2FQ, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.661054908 (14h15m11.176157476s) [noise=true , address=77.90.31.12:4589 , piece_id=L4B3OU3E72GCPP4SZLWPJ2B3LNS6GSFX3FFWQBCVBHL3JGHR7MGA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.662842221 (14h15m11.17794479s) [noise=true , address=77.90.31.19:4841 , piece_id=QV43X4G6B3IXCVDVAJYSGXUUI6LCROUL3IHGNES5JTUEPGKNFA5A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.663362200 (14h15m11.178464776s) [noise=true , address=104.36.82.6:4876 , piece_id=BABG2QF67NQHDH62KKGR2FLZRTF5TWGBNQ5D2PKQL6QJURYIROGA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.666343769 (14h15m11.181446338s) [noise=true , address=104.36.82.16:4294 , piece_id=UHSQZJSGHU3UWAEEOTG5P4NV2YGTGKYYJU7CQ7UC4J6WDUJPUGJA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.675326798 (14h15m11.190429377s) [noise=true , address=104.36.82.2:4730 , piece_id=KY5B7UDNU2S2ZFHCVWZAPJV4QHEMCT4RKKJLOM3SXKTC22HVGKVA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.678428557 (14h15m11.193531125s) [noise=true , address=104.36.82.4:4802 , piece_id=QHJY5W7S3OB2TAIBARTKQTB6HHBRGKPZP2CM7F4V7NYZQLZZSQ7A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.685873433 (14h15m11.200976001s) [noise=true , address=104.36.82.8:4972 , piece_id=F3E53EQXBSUIZDZV5L5CBCYYVE5BOEMSTVEZZMEI7NKKHHKPD2LQ, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.693259479 (14h15m11.208362058s) [noise=true , address=104.36.82.8:4971 , piece_id=HC2FEV5IWEXHECRHUAMAF7J7J4AWYZRTLRD5NUREB7SBCUEA4VGQ, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.693284102 (14h15m11.208386678s) [seg_pos={0 0}, upload=14665]: Segment reached optimal threshold of 13 pieces. Aug 7 23:54:36.693493083 (14h15m11.20859566s) [noise=true , address=104.36.82.7:4909 , piece_id=ISAGUZRBSZZOI5D3XK4N2VBH6CGHWALHK7DUKKDKYTA3PH54WE7A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF Aug 7 23:54:36.693795135 (14h15m11.208897711s) [noise=true , address=104.36.82.5:4837 , piece_id=FJIF5DAJKKHVPDI4FJQFRIY66V6LQXEN7GVMZLCWFPG35QJGSO3Q, seg_pos={0 0}, upload=14665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:55994->104.36.82.5:4837: use of closed network connection; write tcp 159.223.143.39:55994->104.36.82.5:4837: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:55994->104.36.82.5:4837: use of closed network connection Aug 7 23:54:36.693865018 (14h15m11.208967588s) [noise=true , address=104.36.82.13:4185 , piece_id=DY4ZRJG7XBEMN5JTETQR5I6HSLLQNIT3R6ZS7CA5ZBW6QAQVGXSA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: read tcp 159.223.143.39:52730->104.36.82.13:4185: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:54:36.693875333 (14h15m11.208977898s) [seg_pos={0 0}, upload=14665]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:54:36.693954210 (14h15m11.209056786s) [upload=14665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:54:36.693959007 (14h15m11.209061583s) [upload=14665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:36.756106678 (14h15m11.271209244s) [upload=14665]: Done uploading segments... Aug 7 23:54:36.756123317 (14h15m11.271225887s) [upload=14665]: Upload finished. err: <nil> Aug 7 23:55:38.250097734 (14h16m12.765200314s) [upload=14681]: Starting upload Aug 7 23:55:38.250192737 (14h16m12.765295302s) [upload=14681]: Done starting upload Aug 7 23:55:38.250341846 (14h16m12.765444411s) [upload=14681]: Uploading segments... Aug 7 23:55:38.250743926 (14h16m12.76584651s) [upload=14681]: Got next segment. Inline: false Aug 7 23:55:38.250809082 (14h16m12.765911663s) [upload=14681]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:38.250814071 (14h16m12.765916648s) [upload=14681]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:56:08.219284519 (14h16m42.734387085s) [upload=14681]: Next segment err: context deadline exceeded Aug 7 23:56:08.219364566 (14h16m42.734467135s) [upload=14681]: Done uploading segments... Aug 7 23:56:08.219382617 (14h16m42.734485184s) [upload=14681]: Upload finished. err: context deadline exceeded Duration: 407.348993ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 10 out of 60 is failed (threshold=3) [......XX.XXXX...X.X......................................X.X] duration: 1m49.380891839s FAIL (2025-08-07 23:56:11 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== tem: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:21.879557276 (14h14m56.394659852s) [upload=14660]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:54:39.933756270 (14h15m14.44885885s) [seg_pos={0 0}, upload=14660]: Begin upload segment... Aug 7 23:54:39.933829087 (14h15m14.448931664s) [seg_pos={0 0}, upload=14660]: Done begin upload segment. Aug 7 23:54:39.933841087 (14h15m14.448943651s) [noise=true , address=79.127.201.215:4351 , piece_id=VWNGLL7VJ3HPMAPSXYI6QKCNYQTPPLOJSVG7J6KOVLAKKHUCLLXQ, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.933855561 (14h15m14.448958148s) [noise=true , address=79.127.201.215:4349 , piece_id=TDGKTKU6Z4FGNXGPOPZOQXQ32YLYKICAGNB7AG26WHC6LFIWHEPA, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.933931643 (14h15m14.449034228s) [noise=true , address=104.36.82.58:4294 , piece_id=AWHMK4IYW5SOFHLSW5OYVOAT43EV525LGTTZA5NOVUXOHGDFGVZQ, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.934331494 (14h15m14.449434079s) [noise=true , address=77.90.31.17:4789 , piece_id=UAPJNIIAY4C72FKIIGLN7J4H3DHMR64YCY53T5KP2BNAODYYYNTQ, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.934339011 (14h15m14.449441596s) [noise=true , address=104.36.82.23:4447 , piece_id=63VNSCHK7YTAW5X5UWOVUORPY4Z55IJEHYI43LYZMQNINP2VG6CA, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.934712713 (14h15m14.449815294s) [noise=true , address=104.36.82.47:4537 , piece_id=CHRH43QZTM357GGR2EWYK4P5N5POLUQQ6TA67VYEJLWQ75A6YXYQ, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.934726332 (14h15m14.449828913s) [noise=true , address=104.36.82.23:4475 , piece_id=ZKLJCXQBVGOJZSHHAZITBWLTIW3XSA3EXJ5IU35Q3AFGOVUSZMLQ, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.935125663 (14h15m14.450228251s) [noise=true , address=104.36.82.35:4608 , piece_id=EUMFYK2C6SLU5P3L6FBUCWPGBEB27GXJJXU6OBM5DXHUYN3C3NEA, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.935141576 (14h15m14.450244165s) [noise=true , address=104.36.82.5:4850 , piece_id=RVQ3JQ3WI2QTVXI5UQHRCQPIHJB6XBE3XVKBULUEH2JEOK4FZENA, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.935504875 (14h15m14.450607458s) [noise=true , address=104.36.82.9:4040 , piece_id=IT2Y4ZC5F5LBQSYOHZ3YAS5GMMIR4GUFTGJQWOM2ZSNZZ7QR6LJA, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.935516825 (14h15m14.450619411s) [noise=true , address=64.49.13.140:4505 , piece_id=VP5P42UVQ3VAROWEWCUDCQBSAPFI5OYX2XYAYUJWSE7WJIEQVJ6Q, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.935881542 (14h15m14.450984124s) [noise=true , address=104.36.82.10:4097 , piece_id=XL72INQPSMGOCIBWNA2GYUOKLLQKVDAQ27NX5UPZ7Y2WHXIT2SGA, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.935889552 (14h15m14.450992138s) [noise=true , address=45.91.116.7:4040 , piece_id=6NXPRIX6B6LBNGEL7JEFMPET53H5R3A3QNOZRAKDXJCLZPX6KG2A, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.936356864 (14h15m14.451459448s) [noise=true , address=104.36.82.46:4508 , piece_id=HG7TGN3Q5SETJRBS6GTVYJDN5EM7E47XBBLF57PD4TKKNCZCCKIQ, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.938275027 (14h15m14.453377604s) [noise=true , address=45.91.116.4:4966 , piece_id=ZEJLCOYGPIFGTYESTSGTMBW2A5EYKI2BAW5CBM7LWUX2ZVZ2OH7A, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.938941485 (14h15m14.454044056s) [noise=true , address=104.36.82.45:4915 , piece_id=E6ZVUDFXRP5FRVOAAQN2XF2MGVSITTCHGU5X5UVZFZZFLAUF4KNQ, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.939612177 (14h15m14.454714763s) [noise=true , address=104.36.82.18:4370 , piece_id=NTCE3JDSCC5U3TRTIQ7ZWFVR3RE2JQL77GIH23PQWXHXIW2E3TTQ, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:39.939819372 (14h15m14.454921942s) [noise=true , address=45.91.116.8:4059 , piece_id=ALAED7ISKWGLECJEUYVLJOZNNCIVNHUEBYNMBJWALWEJGP2GRCPQ, seg_pos={0 0}, upload=14660]: Uploading piece... Aug 7 23:54:40.038236225 (14h15m14.553338789s) [upload=14660]: Next returned nil segment Aug 7 23:54:40.038258035 (14h15m14.553360621s) [upload=14660]: Waiting for error group managing segments... Aug 7 23:54:40.048242091 (14h15m14.56334466s) [noise=true , address=104.36.82.23:4447 , piece_id=63VNSCHK7YTAW5X5UWOVUORPY4Z55IJEHYI43LYZMQNINP2VG6CA, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.049077306 (14h15m14.564179883s) [noise=true , address=104.36.82.35:4608 , piece_id=EUMFYK2C6SLU5P3L6FBUCWPGBEB27GXJJXU6OBM5DXHUYN3C3NEA, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.051719279 (14h15m14.566821849s) [noise=true , address=104.36.82.18:4370 , piece_id=NTCE3JDSCC5U3TRTIQ7ZWFVR3RE2JQL77GIH23PQWXHXIW2E3TTQ, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.052612115 (14h15m14.567714685s) [noise=true , address=45.91.116.4:4966 , piece_id=ZEJLCOYGPIFGTYESTSGTMBW2A5EYKI2BAW5CBM7LWUX2ZVZ2OH7A, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.060218512 (14h15m14.575321102s) [noise=true , address=45.91.116.8:4059 , piece_id=ALAED7ISKWGLECJEUYVLJOZNNCIVNHUEBYNMBJWALWEJGP2GRCPQ, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.065281928 (14h15m14.580384497s) [noise=true , address=45.91.116.7:4040 , piece_id=6NXPRIX6B6LBNGEL7JEFMPET53H5R3A3QNOZRAKDXJCLZPX6KG2A, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.066022729 (14h15m14.581125298s) [noise=true , address=104.36.82.23:4475 , piece_id=ZKLJCXQBVGOJZSHHAZITBWLTIW3XSA3EXJ5IU35Q3AFGOVUSZMLQ, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.071613288 (14h15m14.586715868s) [noise=true , address=104.36.82.9:4040 , piece_id=IT2Y4ZC5F5LBQSYOHZ3YAS5GMMIR4GUFTGJQWOM2ZSNZZ7QR6LJA, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.071878812 (14h15m14.586981381s) [noise=true , address=104.36.82.10:4097 , piece_id=XL72INQPSMGOCIBWNA2GYUOKLLQKVDAQ27NX5UPZ7Y2WHXIT2SGA, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.074802672 (14h15m14.589905248s) [noise=true , address=77.90.31.17:4789 , piece_id=UAPJNIIAY4C72FKIIGLN7J4H3DHMR64YCY53T5KP2BNAODYYYNTQ, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.082112771 (14h15m14.597215343s) [noise=true , address=104.36.82.5:4850 , piece_id=RVQ3JQ3WI2QTVXI5UQHRCQPIHJB6XBE3XVKBULUEH2JEOK4FZENA, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.207237770 (14h15m14.722340347s) [noise=true , address=79.127.201.215:4351 , piece_id=VWNGLL7VJ3HPMAPSXYI6QKCNYQTPPLOJSVG7J6KOVLAKKHUCLLXQ, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.229458966 (14h15m14.744561535s) [noise=true , address=79.127.201.215:4349 , piece_id=TDGKTKU6Z4FGNXGPOPZOQXQ32YLYKICAGNB7AG26WHC6LFIWHEPA, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.259373189 (14h15m14.774475752s) [noise=true , address=104.36.82.46:4508 , piece_id=HG7TGN3Q5SETJRBS6GTVYJDN5EM7E47XBBLF57PD4TKKNCZCCKIQ, seg_pos={0 0}, upload=14660]: Done uploading piece. err: <nil> Aug 7 23:54:40.259396917 (14h15m14.774499494s) [seg_pos={0 0}, upload=14660]: Segment reached optimal threshold of 14 pieces. Aug 7 23:54:40.260371773 (14h15m14.77547435s) [noise=true , address=104.36.82.45:4915 , piece_id=E6ZVUDFXRP5FRVOAAQN2XF2MGVSITTCHGU5X5UVZFZZFLAUF4KNQ, seg_pos={0 0}, upload=14660]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17rmsv4edxH6jmsN83ceiVJdfEui1qrBK5XBcf7GgvKCn2fknT): protocol: read tcp 159.223.143.39:38678->104.36.82.45:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:54:40.260446597 (14h15m14.775549175s) [noise=true , address=104.36.82.58:4294 , piece_id=AWHMK4IYW5SOFHLSW5OYVOAT43EV525LGTTZA5NOVUXOHGDFGVZQ, seg_pos={0 0}, upload=14660]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12THzQNaXbHuTNCv3CAi8aG31YeFfPW5xe6JveSqKJWyGuHhffW): protocol: read tcp 159.223.143.39:43710->104.36.82.58:4294: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:54:40.260512745 (14h15m14.775615322s) [noise=true , address=104.36.82.47:4537 , piece_id=CHRH43QZTM357GGR2EWYK4P5N5POLUQQ6TA67VYEJLWQ75A6YXYQ, seg_pos={0 0}, upload=14660]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LTTCDNikuwf94KeKsQGFSu6ABXVYt1ombRsEyrZN5JC8d4Eot): protocol: read tcp 159.223.143.39:33428->104.36.82.47:4537: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:54:40.260571145 (14h15m14.775673722s) [noise=true , address=64.49.13.140:4505 , piece_id=VP5P42UVQ3VAROWEWCUDCQBSAPFI5OYX2XYAYUJWSE7WJIEQVJ6Q, seg_pos={0 0}, upload=14660]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1AbFiw33iRaTCNbSt9NmCYG1KBqys85G5VcgXndydkzgnxN4L4): protocol: read tcp 159.223.143.39:34286->64.49.13.140:4505: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:54:40.260583128 (14h15m14.775685695s) [seg_pos={0 0}, upload=14660]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:54:40.260664793 (14h15m14.775767369s) [upload=14660]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:54:40.260669477 (14h15m14.775772054s) [upload=14660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:11.229483942 (14h16m45.744586512s) [upload=14660]: Done uploading segments... Aug 7 23:56:11.229509087 (14h16m45.744611663s) [upload=14660]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m49.38091441s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........X...........................................XXX.X] duration: 30.000898766s FAIL (2025-08-07 23:56:08 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== (14h15m11.016995473s) [upload=14665]: Uploading segments... Aug 7 23:54:36.502140541 (14h15m11.017243126s) [upload=14665]: Got next segment. Inline: false Aug 7 23:54:36.502208316 (14h15m11.017310893s) [upload=14665]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:36.502213415 (14h15m11.017315993s) [upload=14665]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:54:36.540470046 (14h15m11.055572633s) [seg_pos={0 0}, upload=14665]: Begin upload segment... Aug 7 23:54:36.540542869 (14h15m11.055645435s) [seg_pos={0 0}, upload=14665]: Done begin upload segment. Aug 7 23:54:36.540586398 (14h15m11.055688977s) [noise=true , address=77.90.31.10:4520 , piece_id=UXBIKFQ4KOGFTCR73EVF7DBFLAZXOJPPHZUVL4BTJIONLHHQOF3Q, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.540599057 (14h15m11.055701646s) [noise=true , address=104.36.82.7:4909 , piece_id=ISAGUZRBSZZOI5D3XK4N2VBH6CGHWALHK7DUKKDKYTA3PH54WE7A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.541179792 (14h15m11.056282359s) [noise=true , address=45.91.116.6:4975 , piece_id=GSG54APU3E2NKOOHZDOELFVKDEWZPDNYAC6GUW63N2WHS54AHJ5A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.541219031 (14h15m11.056321617s) [noise=true , address=45.91.116.7:4010 , piece_id=HIHBEZ2SFGNUPDJSUPBGYJYIQE5LT5U3KIOWJWTKQWNWHSMB2R6Q, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.541591033 (14h15m11.056693604s) [noise=true , address=104.36.82.5:4837 , piece_id=FJIF5DAJKKHVPDI4FJQFRIY66V6LQXEN7GVMZLCWFPG35QJGSO3Q, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.541808787 (14h15m11.056911375s) [noise=true , address=104.36.82.13:4185 , piece_id=DY4ZRJG7XBEMN5JTETQR5I6HSLLQNIT3R6ZS7CA5ZBW6QAQVGXSA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.542414303 (14h15m11.057516892s) [noise=true , address=104.36.82.16:4294 , piece_id=UHSQZJSGHU3UWAEEOTG5P4NV2YGTGKYYJU7CQ7UC4J6WDUJPUGJA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.542588591 (14h15m11.057691157s) [noise=true , address=104.36.82.4:4802 , piece_id=QHJY5W7S3OB2TAIBARTKQTB6HHBRGKPZP2CM7F4V7NYZQLZZSQ7A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.542987544 (14h15m11.058090129s) [noise=true , address=104.36.82.8:4972 , piece_id=F3E53EQXBSUIZDZV5L5CBCYYVE5BOEMSTVEZZMEI7NKKHHKPD2LQ, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.543563707 (14h15m11.058666294s) [noise=true , address=104.36.82.8:4971 , piece_id=HC2FEV5IWEXHECRHUAMAF7J7J4AWYZRTLRD5NUREB7SBCUEA4VGQ, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.543788500 (14h15m11.058891086s) [noise=true , address=104.36.82.6:4876 , piece_id=BABG2QF67NQHDH62KKGR2FLZRTF5TWGBNQ5D2PKQL6QJURYIROGA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.544354038 (14h15m11.059456606s) [noise=true , address=77.90.31.12:4589 , piece_id=L4B3OU3E72GCPP4SZLWPJ2B3LNS6GSFX3FFWQBCVBHL3JGHR7MGA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.544746164 (14h15m11.059848744s) [noise=true , address=104.36.82.2:4730 , piece_id=KY5B7UDNU2S2ZFHCVWZAPJV4QHEMCT4RKKJLOM3SXKTC22HVGKVA, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.545099366 (14h15m11.060201945s) [noise=true , address=77.90.31.19:4841 , piece_id=QV43X4G6B3IXCVDVAJYSGXUUI6LCROUL3IHGNES5JTUEPGKNFA5A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.545536838 (14h15m11.060639427s) [noise=true , address=77.90.31.14:4663 , piece_id=6OHGW5KCETKDQUXWUUD7RSP6JXHCKTSCPAA22E3QP46OWHE4A2FQ, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.546110762 (14h15m11.061213352s) [noise=true , address=77.90.31.17:4771 , piece_id=EH7SHR3OQ3ODCJ2QKQBVROK3BNG6E4B6PKMTNQMDUA4RRZAGAZ2A, seg_pos={0 0}, upload=14665]: Uploading piece... Aug 7 23:54:36.648423840 (14h15m11.16352641s) [upload=14665]: Next returned nil segment Aug 7 23:54:36.648440535 (14h15m11.163543117s) [upload=14665]: Waiting for error group managing segments... Aug 7 23:54:36.654558819 (14h15m11.169661399s) [noise=true , address=45.91.116.7:4010 , piece_id=HIHBEZ2SFGNUPDJSUPBGYJYIQE5LT5U3KIOWJWTKQWNWHSMB2R6Q, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.655196697 (14h15m11.170299261s) [noise=true , address=77.90.31.10:4520 , piece_id=UXBIKFQ4KOGFTCR73EVF7DBFLAZXOJPPHZUVL4BTJIONLHHQOF3Q, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.658908365 (14h15m11.174010932s) [noise=true , address=77.90.31.17:4771 , piece_id=EH7SHR3OQ3ODCJ2QKQBVROK3BNG6E4B6PKMTNQMDUA4RRZAGAZ2A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.659089278 (14h15m11.174191848s) [noise=true , address=45.91.116.6:4975 , piece_id=GSG54APU3E2NKOOHZDOELFVKDEWZPDNYAC6GUW63N2WHS54AHJ5A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.660774579 (14h15m11.175877148s) [noise=true , address=77.90.31.14:4663 , piece_id=6OHGW5KCETKDQUXWUUD7RSP6JXHCKTSCPAA22E3QP46OWHE4A2FQ, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.661054908 (14h15m11.176157476s) [noise=true , address=77.90.31.12:4589 , piece_id=L4B3OU3E72GCPP4SZLWPJ2B3LNS6GSFX3FFWQBCVBHL3JGHR7MGA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.662842221 (14h15m11.17794479s) [noise=true , address=77.90.31.19:4841 , piece_id=QV43X4G6B3IXCVDVAJYSGXUUI6LCROUL3IHGNES5JTUEPGKNFA5A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.663362200 (14h15m11.178464776s) [noise=true , address=104.36.82.6:4876 , piece_id=BABG2QF67NQHDH62KKGR2FLZRTF5TWGBNQ5D2PKQL6QJURYIROGA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.666343769 (14h15m11.181446338s) [noise=true , address=104.36.82.16:4294 , piece_id=UHSQZJSGHU3UWAEEOTG5P4NV2YGTGKYYJU7CQ7UC4J6WDUJPUGJA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.675326798 (14h15m11.190429377s) [noise=true , address=104.36.82.2:4730 , piece_id=KY5B7UDNU2S2ZFHCVWZAPJV4QHEMCT4RKKJLOM3SXKTC22HVGKVA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.678428557 (14h15m11.193531125s) [noise=true , address=104.36.82.4:4802 , piece_id=QHJY5W7S3OB2TAIBARTKQTB6HHBRGKPZP2CM7F4V7NYZQLZZSQ7A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.685873433 (14h15m11.200976001s) [noise=true , address=104.36.82.8:4972 , piece_id=F3E53EQXBSUIZDZV5L5CBCYYVE5BOEMSTVEZZMEI7NKKHHKPD2LQ, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.693259479 (14h15m11.208362058s) [noise=true , address=104.36.82.8:4971 , piece_id=HC2FEV5IWEXHECRHUAMAF7J7J4AWYZRTLRD5NUREB7SBCUEA4VGQ, seg_pos={0 0}, upload=14665]: Done uploading piece. err: <nil> Aug 7 23:54:36.693284102 (14h15m11.208386678s) [seg_pos={0 0}, upload=14665]: Segment reached optimal threshold of 13 pieces. Aug 7 23:54:36.693493083 (14h15m11.20859566s) [noise=true , address=104.36.82.7:4909 , piece_id=ISAGUZRBSZZOI5D3XK4N2VBH6CGHWALHK7DUKKDKYTA3PH54WE7A, seg_pos={0 0}, upload=14665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF Aug 7 23:54:36.693795135 (14h15m11.208897711s) [noise=true , address=104.36.82.5:4837 , piece_id=FJIF5DAJKKHVPDI4FJQFRIY66V6LQXEN7GVMZLCWFPG35QJGSO3Q, seg_pos={0 0}, upload=14665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:55994->104.36.82.5:4837: use of closed network connection; write tcp 159.223.143.39:55994->104.36.82.5:4837: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:55994->104.36.82.5:4837: use of closed network connection Aug 7 23:54:36.693865018 (14h15m11.208967588s) [noise=true , address=104.36.82.13:4185 , piece_id=DY4ZRJG7XBEMN5JTETQR5I6HSLLQNIT3R6ZS7CA5ZBW6QAQVGXSA, seg_pos={0 0}, upload=14665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: read tcp 159.223.143.39:52730->104.36.82.13:4185: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:54:36.693875333 (14h15m11.208977898s) [seg_pos={0 0}, upload=14665]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:54:36.693954210 (14h15m11.209056786s) [upload=14665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:54:36.693959007 (14h15m11.209061583s) [upload=14665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:36.756106678 (14h15m11.271209244s) [upload=14665]: Done uploading segments... Aug 7 23:54:36.756123317 (14h15m11.271225887s) [upload=14665]: Upload finished. err: <nil> Aug 7 23:55:38.250097734 (14h16m12.765200314s) [upload=14681]: Starting upload Aug 7 23:55:38.250192737 (14h16m12.765295302s) [upload=14681]: Done starting upload Aug 7 23:55:38.250341846 (14h16m12.765444411s) [upload=14681]: Uploading segments... Aug 7 23:55:38.250743926 (14h16m12.76584651s) [upload=14681]: Got next segment. Inline: false Aug 7 23:55:38.250809082 (14h16m12.765911663s) [upload=14681]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:38.250814071 (14h16m12.765916648s) [upload=14681]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:56:08.219284519 (14h16m42.734387085s) [upload=14681]: Next segment err: context deadline exceeded Aug 7 23:56:08.219364566 (14h16m42.734467135s) [upload=14681]: Done uploading segments... Aug 7 23:56:08.219382617 (14h16m42.734485184s) [upload=14681]: Upload finished. err: context deadline exceeded Duration: 30.000924087s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 73.127159ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:56:02 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 73.154244ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [.X.XXX.XX...............................................XX..] duration: 583.179113ms FAIL (2025-08-07 23:55:56 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 583.204431ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [..X.XXX.XX...............................................XX.] duration: 29.071047947s FAIL (2025-08-07 23:55:52 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 29.071059474s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 11 out of 60 is failed (threshold=3) [X..XXXXX.X.XX...........................................X..X] duration: 30.001085118s FAIL (2025-08-07 23:55:48 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001109151s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 29.042264ms target: https://us1.storj.io/login FAIL (2025-08-07 23:55:41 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 29.079288ms core satellite us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [....XXX..X.X.X.....................................X........] duration: 586.106754ms FAIL (2025-08-07 23:55:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 586.132285ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 8 out of 60 is failed (threshold=3) [...XX.XXX..............................................X.XX.] duration: 228.585186ms FAIL (2025-08-07 23:55:36 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== (14h7m29.836816438s) [upload=14530]: Upload finished. err: <nil> Aug 7 23:47:58.537981593 (14h8m33.053084189s) [upload=14545]: Starting upload Aug 7 23:47:58.538091567 (14h8m33.053194153s) [upload=14545]: Done starting upload Aug 7 23:47:58.538175758 (14h8m33.053278336s) [upload=14545]: Uploading segments... Aug 7 23:47:58.538236654 (14h8m33.053339241s) [upload=14545]: Got next segment. Inline: true Aug 7 23:47:58.538243619 (14h8m33.053346194s) [upload=14545]: Waiting for error group managing segments... Aug 7 23:47:58.538296503 (14h8m33.053399195s) [upload=14545]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:58.538302975 (14h8m33.053405563s) [upload=14545]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:58.538308908 (14h8m33.053411487s) [upload=14545]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:58.619189912 (14h8m33.134292494s) [upload=14545]: Done uploading segments... Aug 7 23:47:58.619212540 (14h8m33.134315118s) [upload=14545]: Upload finished. err: <nil> Aug 7 23:49:01.871383066 (14h9m36.386485637s) [upload=14563]: Starting upload Aug 7 23:49:01.871540971 (14h9m36.386643551s) [upload=14563]: Done starting upload Aug 7 23:49:01.871671051 (14h9m36.386773634s) [upload=14563]: Uploading segments... Aug 7 23:49:01.871754766 (14h9m36.386857337s) [upload=14563]: Got next segment. Inline: true Aug 7 23:49:01.871764735 (14h9m36.386867325s) [upload=14563]: Waiting for error group managing segments... Aug 7 23:49:01.871838683 (14h9m36.386941251s) [upload=14563]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:01.871845530 (14h9m36.386948123s) [upload=14563]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:01.871849510 (14h9m36.38695209s) [upload=14563]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:01.934052118 (14h9m36.449154695s) [upload=14563]: Done uploading segments... Aug 7 23:49:01.934071674 (14h9m36.44917425s) [upload=14563]: Upload finished. err: <nil> Aug 7 23:49:30.817755150 (14h10m5.332857717s) [upload=14567]: Starting upload Aug 7 23:49:30.817852337 (14h10m5.332954913s) [upload=14567]: Done starting upload Aug 7 23:49:30.817934904 (14h10m5.333037472s) [upload=14567]: Uploading segments... Aug 7 23:49:30.818001589 (14h10m5.333104172s) [upload=14567]: Got next segment. Inline: true Aug 7 23:49:30.818007295 (14h10m5.333109877s) [upload=14567]: Waiting for error group managing segments... Aug 7 23:49:30.818078537 (14h10m5.333181104s) [upload=14567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:30.818084049 (14h10m5.333186629s) [upload=14567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:30.818087176 (14h10m5.333189753s) [upload=14567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:30.888782953 (14h10m5.403885532s) [upload=14567]: Done uploading segments... Aug 7 23:49:30.888817710 (14h10m5.403920277s) [upload=14567]: Upload finished. err: <nil> Aug 7 23:50:32.085473195 (14h11m6.600575774s) [upload=14591]: Starting upload Aug 7 23:50:32.085550609 (14h11m6.600653185s) [upload=14591]: Done starting upload Aug 7 23:50:32.085610615 (14h11m6.600713183s) [upload=14591]: Uploading segments... Aug 7 23:50:32.085658436 (14h11m6.600761018s) [upload=14591]: Got next segment. Inline: true Aug 7 23:50:32.085662726 (14h11m6.600765303s) [upload=14591]: Waiting for error group managing segments... Aug 7 23:50:32.085716357 (14h11m6.600818923s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:32.085721348 (14h11m6.600823924s) [upload=14591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:32.085724995 (14h11m6.600827572s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:32.142060473 (14h11m6.657163039s) [upload=14591]: Done uploading segments... Aug 7 23:50:32.142076779 (14h11m6.657179345s) [upload=14591]: Upload finished. err: <nil> Aug 7 23:50:57.356595368 (14h11m31.871697934s) [upload=14595]: Starting upload Aug 7 23:50:57.356765546 (14h11m31.871868133s) [upload=14595]: Done starting upload Aug 7 23:50:57.356869337 (14h11m31.871971899s) [upload=14595]: Uploading segments... Aug 7 23:50:57.356914930 (14h11m31.872017522s) [upload=14595]: Got next segment. Inline: true Aug 7 23:50:57.356921513 (14h11m31.872024098s) [upload=14595]: Waiting for error group managing segments... Aug 7 23:50:57.356982351 (14h11m31.872084938s) [upload=14595]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:57.356989548 (14h11m31.872092141s) [upload=14595]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:57.356995954 (14h11m31.872098543s) [upload=14595]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:04.638440451 (14h11m39.153543026s) [upload=14595]: Done uploading segments... Aug 7 23:51:04.638457479 (14h11m39.153560045s) [upload=14595]: Upload finished. err: <nil> Aug 7 23:52:06.228205136 (14h12m40.743307702s) [upload=14620]: Starting upload Aug 7 23:52:06.228273353 (14h12m40.74337593s) [upload=14620]: Done starting upload Aug 7 23:52:06.228361368 (14h12m40.743463938s) [upload=14620]: Uploading segments... Aug 7 23:52:06.228433067 (14h12m40.743535646s) [upload=14620]: Got next segment. Inline: true Aug 7 23:52:06.228437780 (14h12m40.743540356s) [upload=14620]: Waiting for error group managing segments... Aug 7 23:52:06.228479528 (14h12m40.743582104s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:06.228483802 (14h12m40.743586378s) [upload=14620]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:06.228498744 (14h12m40.743601323s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:06.292782592 (14h12m40.807885172s) [upload=14620]: Done uploading segments... Aug 7 23:52:06.292803751 (14h12m40.80790633s) [upload=14620]: Upload finished. err: <nil> Aug 7 23:52:55.204638563 (14h13m29.719741157s) [upload=14636]: Starting upload Aug 7 23:52:55.204832733 (14h13m29.719935326s) [upload=14636]: Done starting upload Aug 7 23:52:55.204913770 (14h13m29.720016348s) [upload=14636]: Uploading segments... Aug 7 23:52:55.205057568 (14h13m29.720160151s) [upload=14636]: Got next segment. Inline: true Aug 7 23:52:55.205064394 (14h13m29.720166971s) [upload=14636]: Waiting for error group managing segments... Aug 7 23:52:55.205114279 (14h13m29.720216864s) [upload=14636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:55.205134068 (14h13m29.720236652s) [upload=14636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:55.205137808 (14h13m29.72024039s) [upload=14636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:25.172932226 (14h13m59.688034802s) [upload=14636]: Done uploading segments... Aug 7 23:53:25.172962711 (14h13m59.688065287s) [upload=14636]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:25.682055912 (14h14m0.197158491s) [upload=14648]: Starting upload Aug 7 23:53:25.682122304 (14h14m0.197224872s) [upload=14648]: Done starting upload Aug 7 23:53:25.682169894 (14h14m0.197272463s) [upload=14648]: Uploading segments... Aug 7 23:53:25.682274124 (14h14m0.197376703s) [upload=14648]: Got next segment. Inline: true Aug 7 23:53:25.682279715 (14h14m0.197382292s) [upload=14648]: Waiting for error group managing segments... Aug 7 23:53:25.682336423 (14h14m0.197438989s) [upload=14648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:25.682341672 (14h14m0.197444249s) [upload=14648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:25.682344934 (14h14m0.197447509s) [upload=14648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:25.755587837 (14h14m0.270690415s) [upload=14648]: Done uploading segments... Aug 7 23:53:25.755603818 (14h14m0.270706384s) [upload=14648]: Upload finished. err: <nil> Aug 7 23:54:03.333646436 (14h14m37.848749023s) [upload=14655]: Starting upload Aug 7 23:54:03.333752884 (14h14m37.848855471s) [upload=14655]: Done starting upload Aug 7 23:54:03.333844300 (14h14m37.848946877s) [upload=14655]: Uploading segments... Aug 7 23:54:03.333909324 (14h14m37.84901191s) [upload=14655]: Got next segment. Inline: true Aug 7 23:54:03.333915381 (14h14m37.849017987s) [upload=14655]: Waiting for error group managing segments... Aug 7 23:54:03.333983662 (14h14m37.849086249s) [upload=14655]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:03.333989573 (14h14m37.849092156s) [upload=14655]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:03.333993282 (14h14m37.849095867s) [upload=14655]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:33.301833586 (14h15m7.816936169s) [upload=14655]: Done uploading segments... Aug 7 23:54:33.301863059 (14h15m7.816965638s) [upload=14655]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:36.259738412 (14h15m10.774840994s) [upload=14664]: Starting upload Aug 7 23:54:36.259845771 (14h15m10.77494834s) [upload=14664]: Done starting upload Aug 7 23:54:36.260065480 (14h15m10.775168051s) [upload=14664]: Uploading segments... Aug 7 23:54:36.260143124 (14h15m10.775245714s) [upload=14664]: Got next segment. Inline: true Aug 7 23:54:36.260148821 (14h15m10.775251404s) [upload=14664]: Waiting for error group managing segments... Aug 7 23:54:36.260217622 (14h15m10.775320206s) [upload=14664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:36.260223500 (14h15m10.77532608s) [upload=14664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:36.260227779 (14h15m10.77533036s) [upload=14664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:06.222352771 (14h15m40.737455348s) [upload=14664]: Done uploading segments... Aug 7 23:55:06.222372750 (14h15m40.737475327s) [upload=14664]: Upload finished. err: metaclient: context deadline exceeded Duration: 228.621695ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 8 out of 60 is failed (threshold=3) [XX.X.X....XX..........................................X..X..] duration: 258.701439ms FAIL (2025-08-07 23:55:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 258.725779ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 6 out of 60 is failed (threshold=3) [...XX..X...X..............................................XX] duration: 1m0.001286926s FAIL (2025-08-07 23:55:22 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Aug 7 23:46:02.612451273 (14h6m37.127553857s) [upload=14516]: Upload finished. err: <nil> Aug 7 23:46:33.174008493 (14h7m7.68911107s) [upload=14523]: Starting upload Aug 7 23:46:33.174095566 (14h7m7.689198146s) [upload=14523]: Done starting upload Aug 7 23:46:33.174169016 (14h7m7.689271584s) [upload=14523]: Uploading segments... Aug 7 23:46:33.174213635 (14h7m7.68931622s) [upload=14523]: Got next segment. Inline: true Aug 7 23:46:33.174220067 (14h7m7.689322652s) [upload=14523]: Waiting for error group managing segments... Aug 7 23:46:33.174271363 (14h7m7.689373942s) [upload=14523]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:46:33.174277423 (14h7m7.689380007s) [upload=14523]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:46:33.174281768 (14h7m7.689384349s) [upload=14523]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:33.410961160 (14h7m7.926063757s) [upload=14523]: Done uploading segments... Aug 7 23:46:33.410982034 (14h7m7.926084613s) [upload=14523]: Upload finished. err: <nil> Aug 7 23:47:43.879457284 (14h8m18.394559871s) [upload=14540]: Starting upload Aug 7 23:47:43.879610635 (14h8m18.394713221s) [upload=14540]: Done starting upload Aug 7 23:47:43.879713914 (14h8m18.394816491s) [upload=14540]: Uploading segments... Aug 7 23:47:43.879808496 (14h8m18.39491109s) [upload=14540]: Got next segment. Inline: true Aug 7 23:47:43.879816280 (14h8m18.394918867s) [upload=14540]: Waiting for error group managing segments... Aug 7 23:47:43.879892463 (14h8m18.394995049s) [upload=14540]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:43.879898862 (14h8m18.395001442s) [upload=14540]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:43.879902786 (14h8m18.395005367s) [upload=14540]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:43.948673039 (14h8m18.463775606s) [upload=14540]: Done uploading segments... Aug 7 23:47:43.948690272 (14h8m18.463792838s) [upload=14540]: Upload finished. err: <nil> Aug 7 23:48:32.733662762 (14h9m7.248765339s) [upload=14555]: Starting upload Aug 7 23:48:32.733764614 (14h9m7.248867195s) [upload=14555]: Done starting upload Aug 7 23:48:32.733839718 (14h9m7.248942295s) [upload=14555]: Uploading segments... Aug 7 23:48:32.733888071 (14h9m7.248990659s) [upload=14555]: Got next segment. Inline: true Aug 7 23:48:32.733893767 (14h9m7.248996349s) [upload=14555]: Waiting for error group managing segments... Aug 7 23:48:32.733943638 (14h9m7.249046215s) [upload=14555]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:32.733949815 (14h9m7.249052399s) [upload=14555]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:32.733954110 (14h9m7.24905669s) [upload=14555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:32.809665685 (14h9m7.324768252s) [upload=14555]: Done uploading segments... Aug 7 23:48:32.809699832 (14h9m7.324802408s) [upload=14555]: Upload finished. err: <nil> Aug 7 23:49:43.381878590 (14h10m17.896981166s) [upload=14573]: Starting upload Aug 7 23:49:43.382020923 (14h10m17.897123495s) [upload=14573]: Done starting upload Aug 7 23:49:43.382128913 (14h10m17.897231484s) [upload=14573]: Uploading segments... Aug 7 23:49:43.382204063 (14h10m17.897306641s) [upload=14573]: Got next segment. Inline: true Aug 7 23:49:43.382211147 (14h10m17.897313718s) [upload=14573]: Waiting for error group managing segments... Aug 7 23:49:43.382287220 (14h10m17.897389786s) [upload=14573]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:43.382292849 (14h10m17.897395421s) [upload=14573]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:43.382297255 (14h10m17.897399833s) [upload=14573]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:43.532367962 (14h10m18.047470542s) [upload=14573]: Done uploading segments... Aug 7 23:49:43.532399853 (14h10m18.047502429s) [upload=14573]: Upload finished. err: <nil> Aug 7 23:50:25.794985700 (14h11m0.310088267s) [upload=14588]: Starting upload Aug 7 23:50:25.795091135 (14h11m0.310193704s) [upload=14588]: Done starting upload Aug 7 23:50:25.795192505 (14h11m0.310295073s) [upload=14588]: Uploading segments... Aug 7 23:50:25.795250709 (14h11m0.310353286s) [upload=14588]: Got next segment. Inline: true Aug 7 23:50:25.795256599 (14h11m0.310359177s) [upload=14588]: Waiting for error group managing segments... Aug 7 23:50:25.795298950 (14h11m0.310401527s) [upload=14588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:25.795303481 (14h11m0.310406058s) [upload=14588]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:25.795306901 (14h11m0.310409476s) [upload=14588]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:25.886991014 (14h11m0.402093596s) [upload=14588]: Done uploading segments... Aug 7 23:50:25.887011651 (14h11m0.402114228s) [upload=14588]: Upload finished. err: <nil> Aug 7 23:51:02.166684989 (14h11m36.681787571s) [upload=14598]: Starting upload Aug 7 23:51:02.166793315 (14h11m36.681895904s) [upload=14598]: Done starting upload Aug 7 23:51:02.167617113 (14h11m36.682719692s) [upload=14598]: Uploading segments... Aug 7 23:51:02.167688699 (14h11m36.682791288s) [upload=14598]: Got next segment. Inline: true Aug 7 23:51:02.167696965 (14h11m36.68279955s) [upload=14598]: Waiting for error group managing segments... Aug 7 23:51:02.167768986 (14h11m36.682871569s) [upload=14598]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:02.167776149 (14h11m36.682878967s) [upload=14598]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:02.167781308 (14h11m36.682883888s) [upload=14598]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:02.249738740 (14h11m36.764841311s) [upload=14598]: Done uploading segments... Aug 7 23:51:02.249754857 (14h11m36.764857435s) [upload=14598]: Upload finished. err: <nil> Aug 7 23:51:48.536453050 (14h12m23.051555623s) [upload=14610]: Starting upload Aug 7 23:51:48.536534615 (14h12m23.051637193s) [upload=14610]: Done starting upload Aug 7 23:51:48.536600634 (14h12m23.051703211s) [upload=14610]: Uploading segments... Aug 7 23:51:48.536652707 (14h12m23.051755293s) [upload=14610]: Got next segment. Inline: true Aug 7 23:51:48.536658284 (14h12m23.051760862s) [upload=14610]: Waiting for error group managing segments... Aug 7 23:51:48.536705902 (14h12m23.051808486s) [upload=14610]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:48.536710850 (14h12m23.051813429s) [upload=14610]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:48.536714377 (14h12m23.051816954s) [upload=14610]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:48.601979182 (14h12m23.117081769s) [upload=14610]: Done uploading segments... Aug 7 23:51:48.602001648 (14h12m23.117104229s) [upload=14610]: Upload finished. err: <nil> Aug 7 23:52:04.519337006 (14h12m39.034439591s) [upload=14619]: Starting upload Aug 7 23:52:04.519460694 (14h12m39.034563258s) [upload=14619]: Done starting upload Aug 7 23:52:04.519569621 (14h12m39.034672186s) [upload=14619]: Uploading segments... Aug 7 23:52:04.519675504 (14h12m39.034778091s) [upload=14619]: Got next segment. Inline: true Aug 7 23:52:04.519701072 (14h12m39.034803649s) [upload=14619]: Waiting for error group managing segments... Aug 7 23:52:04.519755555 (14h12m39.034858131s) [upload=14619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:04.519760650 (14h12m39.034863226s) [upload=14619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:04.519777078 (14h12m39.034879654s) [upload=14619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:04.588422656 (14h12m39.103525241s) [upload=14619]: Done uploading segments... Aug 7 23:52:04.588444719 (14h12m39.103547297s) [upload=14619]: Upload finished. err: <nil> Aug 7 23:53:08.147087231 (14h13m42.662189804s) [upload=14641]: Starting upload Aug 7 23:53:08.147181411 (14h13m42.662283996s) [upload=14641]: Done starting upload Aug 7 23:53:08.147281202 (14h13m42.66238377s) [upload=14641]: Uploading segments... Aug 7 23:53:08.147322830 (14h13m42.662425415s) [upload=14641]: Got next segment. Inline: true Aug 7 23:53:08.147327701 (14h13m42.662430277s) [upload=14641]: Waiting for error group managing segments... Aug 7 23:53:08.147455761 (14h13m42.66255833s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:08.147460709 (14h13m42.662563285s) [upload=14641]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:08.147463890 (14h13m42.662566457s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:08.116969456 (14h14m42.632072039s) [upload=14641]: Done uploading segments... Aug 7 23:54:08.117007036 (14h14m42.632109618s) [upload=14641]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:22.831162336 (14h14m57.346264913s) [upload=14661]: Starting upload Aug 7 23:54:22.831234431 (14h14m57.346337008s) [upload=14661]: Done starting upload Aug 7 23:54:22.831306794 (14h14m57.346409362s) [upload=14661]: Uploading segments... Aug 7 23:54:22.831343257 (14h14m57.346445842s) [upload=14661]: Got next segment. Inline: true Aug 7 23:54:22.831347808 (14h14m57.346450375s) [upload=14661]: Waiting for error group managing segments... Aug 7 23:54:22.831399492 (14h14m57.346502071s) [upload=14661]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:22.831404349 (14h14m57.346506915s) [upload=14661]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:22.831407696 (14h14m57.346510265s) [upload=14661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:22.799123829 (14h15m57.314226409s) [upload=14661]: Done uploading segments... Aug 7 23:55:22.799154191 (14h15m57.314256772s) [upload=14661]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.001308597s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [...X.XXX.XX...............................................XX] duration: 1m0.000354783s FAIL (2025-08-07 23:55:22 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.00036798s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 29.159772ms target: https://us1.storj.io/login FAIL (2025-08-07 23:55:19 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 29.179288ms core satellite us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 48.208628ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:55:09 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 48.236184ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 28.650912ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 23:55:09 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 28.674155ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 8 out of 60 is failed (threshold=3) [....XX.XXX..............................................X.XX] duration: 30.000548853s FAIL (2025-08-07 23:55:06 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== (14h7m29.836816438s) [upload=14530]: Upload finished. err: <nil> Aug 7 23:47:58.537981593 (14h8m33.053084189s) [upload=14545]: Starting upload Aug 7 23:47:58.538091567 (14h8m33.053194153s) [upload=14545]: Done starting upload Aug 7 23:47:58.538175758 (14h8m33.053278336s) [upload=14545]: Uploading segments... Aug 7 23:47:58.538236654 (14h8m33.053339241s) [upload=14545]: Got next segment. Inline: true Aug 7 23:47:58.538243619 (14h8m33.053346194s) [upload=14545]: Waiting for error group managing segments... Aug 7 23:47:58.538296503 (14h8m33.053399195s) [upload=14545]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:58.538302975 (14h8m33.053405563s) [upload=14545]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:58.538308908 (14h8m33.053411487s) [upload=14545]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:58.619189912 (14h8m33.134292494s) [upload=14545]: Done uploading segments... Aug 7 23:47:58.619212540 (14h8m33.134315118s) [upload=14545]: Upload finished. err: <nil> Aug 7 23:49:01.871383066 (14h9m36.386485637s) [upload=14563]: Starting upload Aug 7 23:49:01.871540971 (14h9m36.386643551s) [upload=14563]: Done starting upload Aug 7 23:49:01.871671051 (14h9m36.386773634s) [upload=14563]: Uploading segments... Aug 7 23:49:01.871754766 (14h9m36.386857337s) [upload=14563]: Got next segment. Inline: true Aug 7 23:49:01.871764735 (14h9m36.386867325s) [upload=14563]: Waiting for error group managing segments... Aug 7 23:49:01.871838683 (14h9m36.386941251s) [upload=14563]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:01.871845530 (14h9m36.386948123s) [upload=14563]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:01.871849510 (14h9m36.38695209s) [upload=14563]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:01.934052118 (14h9m36.449154695s) [upload=14563]: Done uploading segments... Aug 7 23:49:01.934071674 (14h9m36.44917425s) [upload=14563]: Upload finished. err: <nil> Aug 7 23:49:30.817755150 (14h10m5.332857717s) [upload=14567]: Starting upload Aug 7 23:49:30.817852337 (14h10m5.332954913s) [upload=14567]: Done starting upload Aug 7 23:49:30.817934904 (14h10m5.333037472s) [upload=14567]: Uploading segments... Aug 7 23:49:30.818001589 (14h10m5.333104172s) [upload=14567]: Got next segment. Inline: true Aug 7 23:49:30.818007295 (14h10m5.333109877s) [upload=14567]: Waiting for error group managing segments... Aug 7 23:49:30.818078537 (14h10m5.333181104s) [upload=14567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:30.818084049 (14h10m5.333186629s) [upload=14567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:30.818087176 (14h10m5.333189753s) [upload=14567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:30.888782953 (14h10m5.403885532s) [upload=14567]: Done uploading segments... Aug 7 23:49:30.888817710 (14h10m5.403920277s) [upload=14567]: Upload finished. err: <nil> Aug 7 23:50:32.085473195 (14h11m6.600575774s) [upload=14591]: Starting upload Aug 7 23:50:32.085550609 (14h11m6.600653185s) [upload=14591]: Done starting upload Aug 7 23:50:32.085610615 (14h11m6.600713183s) [upload=14591]: Uploading segments... Aug 7 23:50:32.085658436 (14h11m6.600761018s) [upload=14591]: Got next segment. Inline: true Aug 7 23:50:32.085662726 (14h11m6.600765303s) [upload=14591]: Waiting for error group managing segments... Aug 7 23:50:32.085716357 (14h11m6.600818923s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:32.085721348 (14h11m6.600823924s) [upload=14591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:32.085724995 (14h11m6.600827572s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:32.142060473 (14h11m6.657163039s) [upload=14591]: Done uploading segments... Aug 7 23:50:32.142076779 (14h11m6.657179345s) [upload=14591]: Upload finished. err: <nil> Aug 7 23:50:57.356595368 (14h11m31.871697934s) [upload=14595]: Starting upload Aug 7 23:50:57.356765546 (14h11m31.871868133s) [upload=14595]: Done starting upload Aug 7 23:50:57.356869337 (14h11m31.871971899s) [upload=14595]: Uploading segments... Aug 7 23:50:57.356914930 (14h11m31.872017522s) [upload=14595]: Got next segment. Inline: true Aug 7 23:50:57.356921513 (14h11m31.872024098s) [upload=14595]: Waiting for error group managing segments... Aug 7 23:50:57.356982351 (14h11m31.872084938s) [upload=14595]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:57.356989548 (14h11m31.872092141s) [upload=14595]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:57.356995954 (14h11m31.872098543s) [upload=14595]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:04.638440451 (14h11m39.153543026s) [upload=14595]: Done uploading segments... Aug 7 23:51:04.638457479 (14h11m39.153560045s) [upload=14595]: Upload finished. err: <nil> Aug 7 23:52:06.228205136 (14h12m40.743307702s) [upload=14620]: Starting upload Aug 7 23:52:06.228273353 (14h12m40.74337593s) [upload=14620]: Done starting upload Aug 7 23:52:06.228361368 (14h12m40.743463938s) [upload=14620]: Uploading segments... Aug 7 23:52:06.228433067 (14h12m40.743535646s) [upload=14620]: Got next segment. Inline: true Aug 7 23:52:06.228437780 (14h12m40.743540356s) [upload=14620]: Waiting for error group managing segments... Aug 7 23:52:06.228479528 (14h12m40.743582104s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:06.228483802 (14h12m40.743586378s) [upload=14620]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:06.228498744 (14h12m40.743601323s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:06.292782592 (14h12m40.807885172s) [upload=14620]: Done uploading segments... Aug 7 23:52:06.292803751 (14h12m40.80790633s) [upload=14620]: Upload finished. err: <nil> Aug 7 23:52:55.204638563 (14h13m29.719741157s) [upload=14636]: Starting upload Aug 7 23:52:55.204832733 (14h13m29.719935326s) [upload=14636]: Done starting upload Aug 7 23:52:55.204913770 (14h13m29.720016348s) [upload=14636]: Uploading segments... Aug 7 23:52:55.205057568 (14h13m29.720160151s) [upload=14636]: Got next segment. Inline: true Aug 7 23:52:55.205064394 (14h13m29.720166971s) [upload=14636]: Waiting for error group managing segments... Aug 7 23:52:55.205114279 (14h13m29.720216864s) [upload=14636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:55.205134068 (14h13m29.720236652s) [upload=14636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:55.205137808 (14h13m29.72024039s) [upload=14636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:25.172932226 (14h13m59.688034802s) [upload=14636]: Done uploading segments... Aug 7 23:53:25.172962711 (14h13m59.688065287s) [upload=14636]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:25.682055912 (14h14m0.197158491s) [upload=14648]: Starting upload Aug 7 23:53:25.682122304 (14h14m0.197224872s) [upload=14648]: Done starting upload Aug 7 23:53:25.682169894 (14h14m0.197272463s) [upload=14648]: Uploading segments... Aug 7 23:53:25.682274124 (14h14m0.197376703s) [upload=14648]: Got next segment. Inline: true Aug 7 23:53:25.682279715 (14h14m0.197382292s) [upload=14648]: Waiting for error group managing segments... Aug 7 23:53:25.682336423 (14h14m0.197438989s) [upload=14648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:25.682341672 (14h14m0.197444249s) [upload=14648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:25.682344934 (14h14m0.197447509s) [upload=14648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:25.755587837 (14h14m0.270690415s) [upload=14648]: Done uploading segments... Aug 7 23:53:25.755603818 (14h14m0.270706384s) [upload=14648]: Upload finished. err: <nil> Aug 7 23:54:03.333646436 (14h14m37.848749023s) [upload=14655]: Starting upload Aug 7 23:54:03.333752884 (14h14m37.848855471s) [upload=14655]: Done starting upload Aug 7 23:54:03.333844300 (14h14m37.848946877s) [upload=14655]: Uploading segments... Aug 7 23:54:03.333909324 (14h14m37.84901191s) [upload=14655]: Got next segment. Inline: true Aug 7 23:54:03.333915381 (14h14m37.849017987s) [upload=14655]: Waiting for error group managing segments... Aug 7 23:54:03.333983662 (14h14m37.849086249s) [upload=14655]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:03.333989573 (14h14m37.849092156s) [upload=14655]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:03.333993282 (14h14m37.849095867s) [upload=14655]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:33.301833586 (14h15m7.816936169s) [upload=14655]: Done uploading segments... Aug 7 23:54:33.301863059 (14h15m7.816965638s) [upload=14655]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:36.259738412 (14h15m10.774840994s) [upload=14664]: Starting upload Aug 7 23:54:36.259845771 (14h15m10.77494834s) [upload=14664]: Done starting upload Aug 7 23:54:36.260065480 (14h15m10.775168051s) [upload=14664]: Uploading segments... Aug 7 23:54:36.260143124 (14h15m10.775245714s) [upload=14664]: Got next segment. Inline: true Aug 7 23:54:36.260148821 (14h15m10.775251404s) [upload=14664]: Waiting for error group managing segments... Aug 7 23:54:36.260217622 (14h15m10.775320206s) [upload=14664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:36.260223500 (14h15m10.77532608s) [upload=14664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:36.260227779 (14h15m10.77533036s) [upload=14664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:06.222352771 (14h15m40.737455348s) [upload=14664]: Done uploading segments... Aug 7 23:55:06.222372750 (14h15m40.737475327s) [upload=14664]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000572627s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....XXX..X.X.X.....................................X.......] duration: 503.194628ms FAIL (2025-08-07 23:55:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 503.22307ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 8 out of 60 is failed (threshold=3) [.XX.X.X....XX..........................................X..X.] duration: 267.53558ms FAIL (2025-08-07 23:55:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 267.639696ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 8 out of 60 is failed (threshold=3) [..XX.X.X....XX..........................................X..X] duration: 30.00099281s FAIL (2025-08-07 23:54:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00103307s edge gateway us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 31.817193ms target: https://us1.storj.io/login FAIL (2025-08-07 23:54:55 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 31.841231ms core satellite us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 209.325269ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 23:54:37 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 209.349906ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............X...........................................XXX.] duration: 500.071989ms FAIL (2025-08-07 23:54:36 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 87s) [noise=true , address=104.36.82.13:4186 , piece_id=QAOLE7WMNTFWJYCX44POJU2KE4EH65OXAXCFBI2CLP5IHUPPT5VA, seg_pos={0 0}, upload=14606]: Uploading piece... Aug 7 23:51:35.406068948 (14h12m9.921171527s) [noise=true , address=77.90.31.13:4628 , piece_id=4KYOK67JOALTBRXBHBYDWBDG75HZQAZVRWYM2ECKDKZZ3SXKP3XQ, seg_pos={0 0}, upload=14606]: Uploading piece... Aug 7 23:51:35.406318146 (14h12m9.921420725s) [noise=true , address=104.36.82.14:4221 , piece_id=KYCQMYOIN3MV5JW63KCLACFFDQWYMJZTRVEUOVFGCX3XWLRI5DGQ, seg_pos={0 0}, upload=14606]: Uploading piece... Aug 7 23:51:35.406682793 (14h12m9.92178538s) [noise=true , address=104.36.82.8:4971 , piece_id=ZC54WOM3C7LNVM2X6P5I2GPGZ7IZTORQ2DYUYDM7US3GKOYGQRHA, seg_pos={0 0}, upload=14606]: Uploading piece... Aug 7 23:51:35.407100619 (14h12m9.922203195s) [noise=true , address=104.36.82.16:4294 , piece_id=XIHQZDR5IMQQMZTO75Q7KKUM5ZR6532ABE3TI55JB4YALRWF4T5Q, seg_pos={0 0}, upload=14606]: Uploading piece... Aug 7 23:51:35.407521118 (14h12m9.922623705s) [noise=true , address=45.91.116.8:4047 , piece_id=34GCHEBUJ2PX7AJ5A2NQQSGMDX6PNUMTCMCPXTFMFW3GDUKKFWBQ, seg_pos={0 0}, upload=14606]: Uploading piece... Aug 7 23:51:35.408541750 (14h12m9.92364433s) [noise=true , address=104.36.82.15:4258 , piece_id=CE5COWKW32DRRZBBMT4RJKLL3Q6PANJYQDRR524YYWNLFZDE3YVA, seg_pos={0 0}, upload=14606]: Uploading piece... Aug 7 23:51:35.408920069 (14h12m9.924022648s) [noise=true , address=45.91.116.13:4155 , piece_id=3XXELTRHRR5RDDEYHV5TBNEUPUTIY44CKIAH5N7KTTKBZ3GNFYEQ, seg_pos={0 0}, upload=14606]: Uploading piece... Aug 7 23:51:35.409269498 (14h12m9.924372079s) [noise=true , address=104.36.82.12:4151 , piece_id=FQ7RTJ6RSKQXLY2QLWMPZNBFLQOH7RYV2QMDM247YKPOQZ43IJXQ, seg_pos={0 0}, upload=14606]: Uploading piece... Aug 7 23:51:35.412783363 (14h12m9.927885934s) [noise=true , address=77.90.31.14:4661 , piece_id=2UBBTMXSKDSUCA7SCOMGERAWBHDNIMZXP6PTJPZ2WN76KIUGATIA, seg_pos={0 0}, upload=14606]: Uploading piece... Aug 7 23:51:35.524193177 (14h12m10.039295755s) [upload=14606]: Next returned nil segment Aug 7 23:51:35.524218480 (14h12m10.039321059s) [upload=14606]: Waiting for error group managing segments... Aug 7 23:51:35.528680921 (14h12m10.04378349s) [noise=true , address=77.90.31.14:4661 , piece_id=2UBBTMXSKDSUCA7SCOMGERAWBHDNIMZXP6PTJPZ2WN76KIUGATIA, seg_pos={0 0}, upload=14606]: Done uploading piece. err: <nil> Aug 7 23:51:35.529111404 (14h12m10.044213966s) [noise=true , address=45.91.116.8:4047 , piece_id=34GCHEBUJ2PX7AJ5A2NQQSGMDX6PNUMTCMCPXTFMFW3GDUKKFWBQ, seg_pos={0 0}, upload=14606]: Done uploading piece. err: <nil> Aug 7 23:51:35.529204740 (14h12m10.044307308s) [noise=true , address=45.91.116.12:4225 , piece_id=3223EE4DK2F7PMVC57QVG4IB5SK6VBZOP4OXEIZTAR23DH4N3FJQ, seg_pos={0 0}, upload=14606]: Done uploading piece. err: <nil> Aug 7 23:51:35.531697560 (14h12m10.046800142s) [noise=true , address=77.90.31.13:4628 , piece_id=4KYOK67JOALTBRXBHBYDWBDG75HZQAZVRWYM2ECKDKZZ3SXKP3XQ, seg_pos={0 0}, upload=14606]: Done uploading piece. err: <nil> Aug 7 23:51:35.532347107 (14h12m10.047449676s) [noise=true , address=45.91.116.4:4939 , piece_id=H6RWITCLY42NEGDEDTBG657Y3PB2OAJQ6KEK7C3WLFLQZRP6WU3Q, seg_pos={0 0}, upload=14606]: Done uploading piece. err: <nil> Aug 7 23:51:35.532868683 (14h12m10.047971245s) [noise=true , address=45.91.116.13:4155 , piece_id=3XXELTRHRR5RDDEYHV5TBNEUPUTIY44CKIAH5N7KTTKBZ3GNFYEQ, seg_pos={0 0}, upload=14606]: Done uploading piece. err: <nil> Aug 7 23:51:35.533320354 (14h12m10.048422922s) [noise=true , address=77.90.31.11:4555 , piece_id=6T4BNHJ65FRZ75LLIJ6KBKTIJYO7IGFBLRHMHS7U5LYSZ2SRNBTQ, seg_pos={0 0}, upload=14606]: Done uploading piece. err: <nil> Aug 7 23:51:35.538791806 (14h12m10.053894368s) [noise=true , address=104.36.82.15:4259 , piece_id=PGSORNWLXW6RC74ATT3NFSY6VC7W4TFLO2LAMGUYV3RMRVWPMNQQ, seg_pos={0 0}, upload=14606]: Done uploading piece. err: <nil> Aug 7 23:51:35.539514067 (14h12m10.054616646s) [noise=true , address=104.36.82.2:4732 , piece_id=2GK4RULPRTBTX7LSEYQUT4UR6B5AYJSYNQAA5YDCCBSIRAUPLTOA, seg_pos={0 0}, upload=14606]: Done uploading piece. err: <nil> Aug 7 23:51:35.539606529 (14h12m10.054709099s) [noise=true , address=104.36.82.16:4294 , piece_id=XIHQZDR5IMQQMZTO75Q7KKUM5ZR6532ABE3TI55JB4YALRWF4T5Q, seg_pos={0 0}, upload=14606]: Done uploading piece. err: <nil> Aug 7 23:51:35.540330858 (14h12m10.055433492s) [noise=true , address=104.36.82.5:4839 , piece_id=42C3BS7PSBMETSEEP3GWSOH4WUT2JUHH5QGBNZ5TAANW4WDOGH2Q, seg_pos={0 0}, upload=14606]: Done uploading piece. err: <nil> Aug 7 23:51:35.543245378 (14h12m10.058347955s) [noise=true , address=104.36.82.12:4151 , piece_id=FQ7RTJ6RSKQXLY2QLWMPZNBFLQOH7RYV2QMDM247YKPOQZ43IJXQ, seg_pos={0 0}, upload=14606]: Done uploading piece. err: <nil> Aug 7 23:51:35.557579820 (14h12m10.072682407s) [noise=true , address=104.36.82.14:4221 , piece_id=KYCQMYOIN3MV5JW63KCLACFFDQWYMJZTRVEUOVFGCX3XWLRI5DGQ, seg_pos={0 0}, upload=14606]: Done uploading piece. err: <nil> Aug 7 23:51:35.557631566 (14h12m10.072734144s) [seg_pos={0 0}, upload=14606]: Segment reached optimal threshold of 13 pieces. Aug 7 23:51:35.558140786 (14h12m10.073243372s) [noise=true , address=104.36.82.8:4971 , piece_id=ZC54WOM3C7LNVM2X6P5I2GPGZ7IZTORQ2DYUYDM7US3GKOYGQRHA, seg_pos={0 0}, upload=14606]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 159.223.143.39:58716->104.36.82.8:4971: use of closed network connection; write tcp 159.223.143.39:58716->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:58716->104.36.82.8:4971: use of closed network connection Aug 7 23:51:35.558325994 (14h12m10.07342858s) [noise=true , address=104.36.82.15:4258 , piece_id=CE5COWKW32DRRZBBMT4RJKLL3Q6PANJYQDRR524YYWNLFZDE3YVA, seg_pos={0 0}, upload=14606]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: read tcp 159.223.143.39:33696->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:51:35.558419532 (14h12m10.0735221s) [noise=true , address=104.36.82.13:4186 , piece_id=QAOLE7WMNTFWJYCX44POJU2KE4EH65OXAXCFBI2CLP5IHUPPT5VA, seg_pos={0 0}, upload=14606]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: read tcp 159.223.143.39:41094->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:51:35.558451109 (14h12m10.073553694s) [seg_pos={0 0}, upload=14606]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:51:35.558580243 (14h12m10.073682807s) [upload=14606]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:51:35.558586543 (14h12m10.073689127s) [upload=14606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:35.707214657 (14h12m10.22231724s) [upload=14606]: Done uploading segments... Aug 7 23:51:35.707291109 (14h12m10.222393688s) [upload=14606]: Upload finished. err: <nil> Aug 7 23:52:35.986308779 (14h13m10.501411356s) [upload=14629]: Starting upload Aug 7 23:52:35.986399932 (14h13m10.501502509s) [upload=14629]: Done starting upload Aug 7 23:52:35.986490908 (14h13m10.501593485s) [upload=14629]: Uploading segments... Aug 7 23:52:35.986775485 (14h13m10.501878054s) [upload=14629]: Got next segment. Inline: false Aug 7 23:52:35.986847602 (14h13m10.501950179s) [upload=14629]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:35.986869666 (14h13m10.501972243s) [upload=14629]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:53:05.954641948 (14h13m40.469744682s) [upload=14629]: Next segment err: context deadline exceeded Aug 7 23:53:05.954726017 (14h13m40.469828594s) [upload=14629]: Done uploading segments... Aug 7 23:53:05.954747075 (14h13m40.469849637s) [upload=14629]: Upload finished. err: context deadline exceeded Aug 7 23:53:06.053410441 (14h13m40.568513021s) [upload=14639]: Starting upload Aug 7 23:53:06.053513846 (14h13m40.568616435s) [upload=14639]: Done starting upload Aug 7 23:53:06.053655272 (14h13m40.568757841s) [upload=14639]: Uploading segments... Aug 7 23:53:06.053907022 (14h13m40.569009591s) [upload=14639]: Got next segment. Inline: false Aug 7 23:53:06.053961798 (14h13m40.569064377s) [upload=14639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:06.053967636 (14h13m40.569070215s) [upload=14639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:53:36.020489094 (14h14m10.535591683s) [upload=14639]: Next segment err: context deadline exceeded Aug 7 23:53:36.020570412 (14h14m10.535672979s) [upload=14639]: Done uploading segments... Aug 7 23:53:36.020593471 (14h14m10.535696042s) [upload=14639]: Upload finished. err: context deadline exceeded Aug 7 23:54:06.432308463 (14h14m40.947411043s) [upload=14657]: Starting upload Aug 7 23:54:06.432379062 (14h14m40.947481647s) [upload=14657]: Done starting upload Aug 7 23:54:06.432611740 (14h14m40.947714325s) [upload=14657]: Uploading segments... Aug 7 23:54:06.432902858 (14h14m40.948005426s) [upload=14657]: Got next segment. Inline: false Aug 7 23:54:06.432946422 (14h14m40.948048988s) [upload=14657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:06.432962156 (14h14m40.948064732s) [upload=14657]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:54:36.397880635 (14h15m10.912983201s) [upload=14657]: Next segment err: context deadline exceeded Aug 7 23:54:36.397964918 (14h15m10.913067484s) [upload=14657]: Done uploading segments... Aug 7 23:54:36.397981679 (14h15m10.913084245s) [upload=14657]: Upload finished. err: context deadline exceeded Duration: 500.09215ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 7 out of 60 is failed (threshold=3) [.....XX.XXX..............................................X.X] duration: 30.001016568s FAIL (2025-08-07 23:54:33 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ng segments... Aug 7 23:45:44.774865128 (14h6m19.289967705s) [upload=14512]: Upload finished. err: <nil> Aug 7 23:46:55.221569882 (14h7m29.736672449s) [upload=14530]: Starting upload Aug 7 23:46:55.221642729 (14h7m29.736745314s) [upload=14530]: Done starting upload Aug 7 23:46:55.221734906 (14h7m29.736837478s) [upload=14530]: Uploading segments... Aug 7 23:46:55.221804000 (14h7m29.736906588s) [upload=14530]: Got next segment. Inline: true Aug 7 23:46:55.221811160 (14h7m29.736913729s) [upload=14530]: Waiting for error group managing segments... Aug 7 23:46:55.221885036 (14h7m29.7369876s) [upload=14530]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:46:55.221890384 (14h7m29.736992966s) [upload=14530]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:46:55.221893621 (14h7m29.736996201s) [upload=14530]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:55.321692569 (14h7m29.836795155s) [upload=14530]: Done uploading segments... Aug 7 23:46:55.321713851 (14h7m29.836816438s) [upload=14530]: Upload finished. err: <nil> Aug 7 23:47:58.537981593 (14h8m33.053084189s) [upload=14545]: Starting upload Aug 7 23:47:58.538091567 (14h8m33.053194153s) [upload=14545]: Done starting upload Aug 7 23:47:58.538175758 (14h8m33.053278336s) [upload=14545]: Uploading segments... Aug 7 23:47:58.538236654 (14h8m33.053339241s) [upload=14545]: Got next segment. Inline: true Aug 7 23:47:58.538243619 (14h8m33.053346194s) [upload=14545]: Waiting for error group managing segments... Aug 7 23:47:58.538296503 (14h8m33.053399195s) [upload=14545]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:58.538302975 (14h8m33.053405563s) [upload=14545]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:58.538308908 (14h8m33.053411487s) [upload=14545]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:58.619189912 (14h8m33.134292494s) [upload=14545]: Done uploading segments... Aug 7 23:47:58.619212540 (14h8m33.134315118s) [upload=14545]: Upload finished. err: <nil> Aug 7 23:49:01.871383066 (14h9m36.386485637s) [upload=14563]: Starting upload Aug 7 23:49:01.871540971 (14h9m36.386643551s) [upload=14563]: Done starting upload Aug 7 23:49:01.871671051 (14h9m36.386773634s) [upload=14563]: Uploading segments... Aug 7 23:49:01.871754766 (14h9m36.386857337s) [upload=14563]: Got next segment. Inline: true Aug 7 23:49:01.871764735 (14h9m36.386867325s) [upload=14563]: Waiting for error group managing segments... Aug 7 23:49:01.871838683 (14h9m36.386941251s) [upload=14563]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:01.871845530 (14h9m36.386948123s) [upload=14563]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:01.871849510 (14h9m36.38695209s) [upload=14563]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:01.934052118 (14h9m36.449154695s) [upload=14563]: Done uploading segments... Aug 7 23:49:01.934071674 (14h9m36.44917425s) [upload=14563]: Upload finished. err: <nil> Aug 7 23:49:30.817755150 (14h10m5.332857717s) [upload=14567]: Starting upload Aug 7 23:49:30.817852337 (14h10m5.332954913s) [upload=14567]: Done starting upload Aug 7 23:49:30.817934904 (14h10m5.333037472s) [upload=14567]: Uploading segments... Aug 7 23:49:30.818001589 (14h10m5.333104172s) [upload=14567]: Got next segment. Inline: true Aug 7 23:49:30.818007295 (14h10m5.333109877s) [upload=14567]: Waiting for error group managing segments... Aug 7 23:49:30.818078537 (14h10m5.333181104s) [upload=14567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:30.818084049 (14h10m5.333186629s) [upload=14567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:30.818087176 (14h10m5.333189753s) [upload=14567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:30.888782953 (14h10m5.403885532s) [upload=14567]: Done uploading segments... Aug 7 23:49:30.888817710 (14h10m5.403920277s) [upload=14567]: Upload finished. err: <nil> Aug 7 23:50:32.085473195 (14h11m6.600575774s) [upload=14591]: Starting upload Aug 7 23:50:32.085550609 (14h11m6.600653185s) [upload=14591]: Done starting upload Aug 7 23:50:32.085610615 (14h11m6.600713183s) [upload=14591]: Uploading segments... Aug 7 23:50:32.085658436 (14h11m6.600761018s) [upload=14591]: Got next segment. Inline: true Aug 7 23:50:32.085662726 (14h11m6.600765303s) [upload=14591]: Waiting for error group managing segments... Aug 7 23:50:32.085716357 (14h11m6.600818923s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:32.085721348 (14h11m6.600823924s) [upload=14591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:32.085724995 (14h11m6.600827572s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:32.142060473 (14h11m6.657163039s) [upload=14591]: Done uploading segments... Aug 7 23:50:32.142076779 (14h11m6.657179345s) [upload=14591]: Upload finished. err: <nil> Aug 7 23:50:57.356595368 (14h11m31.871697934s) [upload=14595]: Starting upload Aug 7 23:50:57.356765546 (14h11m31.871868133s) [upload=14595]: Done starting upload Aug 7 23:50:57.356869337 (14h11m31.871971899s) [upload=14595]: Uploading segments... Aug 7 23:50:57.356914930 (14h11m31.872017522s) [upload=14595]: Got next segment. Inline: true Aug 7 23:50:57.356921513 (14h11m31.872024098s) [upload=14595]: Waiting for error group managing segments... Aug 7 23:50:57.356982351 (14h11m31.872084938s) [upload=14595]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:57.356989548 (14h11m31.872092141s) [upload=14595]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:57.356995954 (14h11m31.872098543s) [upload=14595]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:04.638440451 (14h11m39.153543026s) [upload=14595]: Done uploading segments... Aug 7 23:51:04.638457479 (14h11m39.153560045s) [upload=14595]: Upload finished. err: <nil> Aug 7 23:52:06.228205136 (14h12m40.743307702s) [upload=14620]: Starting upload Aug 7 23:52:06.228273353 (14h12m40.74337593s) [upload=14620]: Done starting upload Aug 7 23:52:06.228361368 (14h12m40.743463938s) [upload=14620]: Uploading segments... Aug 7 23:52:06.228433067 (14h12m40.743535646s) [upload=14620]: Got next segment. Inline: true Aug 7 23:52:06.228437780 (14h12m40.743540356s) [upload=14620]: Waiting for error group managing segments... Aug 7 23:52:06.228479528 (14h12m40.743582104s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:06.228483802 (14h12m40.743586378s) [upload=14620]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:06.228498744 (14h12m40.743601323s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:06.292782592 (14h12m40.807885172s) [upload=14620]: Done uploading segments... Aug 7 23:52:06.292803751 (14h12m40.80790633s) [upload=14620]: Upload finished. err: <nil> Aug 7 23:52:55.204638563 (14h13m29.719741157s) [upload=14636]: Starting upload Aug 7 23:52:55.204832733 (14h13m29.719935326s) [upload=14636]: Done starting upload Aug 7 23:52:55.204913770 (14h13m29.720016348s) [upload=14636]: Uploading segments... Aug 7 23:52:55.205057568 (14h13m29.720160151s) [upload=14636]: Got next segment. Inline: true Aug 7 23:52:55.205064394 (14h13m29.720166971s) [upload=14636]: Waiting for error group managing segments... Aug 7 23:52:55.205114279 (14h13m29.720216864s) [upload=14636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:55.205134068 (14h13m29.720236652s) [upload=14636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:55.205137808 (14h13m29.72024039s) [upload=14636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:25.172932226 (14h13m59.688034802s) [upload=14636]: Done uploading segments... Aug 7 23:53:25.172962711 (14h13m59.688065287s) [upload=14636]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:25.682055912 (14h14m0.197158491s) [upload=14648]: Starting upload Aug 7 23:53:25.682122304 (14h14m0.197224872s) [upload=14648]: Done starting upload Aug 7 23:53:25.682169894 (14h14m0.197272463s) [upload=14648]: Uploading segments... Aug 7 23:53:25.682274124 (14h14m0.197376703s) [upload=14648]: Got next segment. Inline: true Aug 7 23:53:25.682279715 (14h14m0.197382292s) [upload=14648]: Waiting for error group managing segments... Aug 7 23:53:25.682336423 (14h14m0.197438989s) [upload=14648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:25.682341672 (14h14m0.197444249s) [upload=14648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:25.682344934 (14h14m0.197447509s) [upload=14648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:25.755587837 (14h14m0.270690415s) [upload=14648]: Done uploading segments... Aug 7 23:53:25.755603818 (14h14m0.270706384s) [upload=14648]: Upload finished. err: <nil> Aug 7 23:54:03.333646436 (14h14m37.848749023s) [upload=14655]: Starting upload Aug 7 23:54:03.333752884 (14h14m37.848855471s) [upload=14655]: Done starting upload Aug 7 23:54:03.333844300 (14h14m37.848946877s) [upload=14655]: Uploading segments... Aug 7 23:54:03.333909324 (14h14m37.84901191s) [upload=14655]: Got next segment. Inline: true Aug 7 23:54:03.333915381 (14h14m37.849017987s) [upload=14655]: Waiting for error group managing segments... Aug 7 23:54:03.333983662 (14h14m37.849086249s) [upload=14655]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:03.333989573 (14h14m37.849092156s) [upload=14655]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:03.333993282 (14h14m37.849095867s) [upload=14655]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:33.301833586 (14h15m7.816936169s) [upload=14655]: Done uploading segments... Aug 7 23:54:33.301863059 (14h15m7.816965638s) [upload=14655]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.001043077s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [....X.XXX.XX...............................................X] duration: 1m0.001286519s FAIL (2025-08-07 23:54:22 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001306819s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [......XXX..X.X.X.....................................X......] duration: 529.041961ms FAIL (2025-08-07 23:54:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 529.652918ms edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 46.821468ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:54:20 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 46.843154ms edge gateway vhost us-select-1

Orbiter Check History

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)