Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
12.929554ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Length=274 Content-Type=application/xml Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=18981B024774DCCC X-Request-Id=VVntfFR3K88 Date=Fri, 27 Feb 2026 12:42:29 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/1507b0ff4a8571ec
FAIL (2026-02-27 12:42:29 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981B024774DCCC</RequestId><HostId></HostId></Error>"
Duration: 12.955655ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
27.361266ms remote:
"tcp:45.38.24.2:443" response headers:
Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Vary=Origin X-Amz-Request-Id=18981B01A9396271 X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:42:27 GMT Content-Length=284 Content-Type=application/xml Server=Storj X-Request-Id=R9cxKsPMwH9 target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/73119b4be8f4dfad
FAIL (2026-02-27 12:42:27 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981B01A9396271</RequestId><HostId></HostId></Error>"
Duration: 27.383811ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.723948ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Type=application/xml Server=Storj X-Amz-Request-Id=18981AFF2040A5C3 X-Request-Id=bKTuma6wHYx Content-Length=274 Vary=Origin X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:42:16 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/746208257478ec93
FAIL (2026-02-27 12:42:16 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AFF2040A5C3</RequestId><HostId></HostId></Error>"
Duration: 9.749796ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
29.12344ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Length=284 Content-Security-Policy=block-all-mixed-content Vary=Origin Date=Fri, 27 Feb 2026 12:42:07 GMT Accept-Ranges=bytes Content-Type=application/xml Server=Storj X-Amz-Request-Id=18981AFCF9B8991F X-Request-Id=A8MVvppyUAN X-Xss-Protection=1; mode=block target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/6f01d185fcf565e9
FAIL (2026-02-27 12:42:07 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AFCF9B8991F</RequestId><HostId></HostId></Error>"
Duration: 29.149605ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
12.935949ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml Date=Fri, 27 Feb 2026 12:42:02 GMT Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Amz-Request-Id=18981AFBE9C6E7E2 X-Request-Id=gCyroiuGWhf X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/68fdc05d5c772dcf
FAIL (2026-02-27 12:42:02 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AFBE9C6E7E2</RequestId><HostId></HostId></Error>"
Duration: 12.964176ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
1.237306773s trace:
https://jaeger.dev.storj.io/trace/41604b8e737e0644
FAIL (2026-02-27 12:41:50 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
Feb 27 12:24:12.958062237 (232h8m13.761421427s) [upload=241649]: Starting upload
Feb 27 12:24:12.958140898 (232h8m13.761500081s) [upload=241649]: Done starting upload
Feb 27 12:24:12.958361897 (232h8m13.761721063s) [upload=241649]: Uploading segments...
Feb 27 12:24:12.958461336 (232h8m13.761820517s) [upload=241649]: Got next segment. Inline: false
Feb 27 12:24:12.958521453 (232h8m13.761880637s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 27 12:24:12.958527047 (232h8m13.761886222s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 27 12:24:12.996321566 (232h8m13.79968075s) [seg_pos={0 0}, upload=241649]: Begin upload segment...
Feb 27 12:24:12.996448820 (232h8m13.799807999s) [seg_pos={0 0}, upload=241649]: Done begin upload segment.
Feb 27 12:24:12.996528741 (232h8m13.799887926s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.996657536 (232h8m13.800016719s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.998274501 (232h8m13.801633687s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999072752 (232h8m13.802431938s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999746928 (232h8m13.803106112s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000114348 (232h8m13.803473543s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000906074 (232h8m13.804265302s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000989520 (232h8m13.804348704s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001455955 (232h8m13.804815197s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001604735 (232h8m13.804963973s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002253164 (232h8m13.805612348s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002863992 (232h8m13.806223173s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003499182 (232h8m13.806858378s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003922169 (232h8m13.807281352s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004123035 (232h8m13.807482216s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004561506 (232h8m13.807920691s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.005503797 (232h8m13.808862985s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.006763318 (232h8m13.810122494s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.096582917 (232h8m13.899942093s) [upload=241649]: Next returned nil segment
Feb 27 12:24:13.096600442 (232h8m13.899959621s) [upload=241649]: Waiting for error group managing segments...
Feb 27 12:24:13.100656351 (232h8m13.904015632s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.105829543 (232h8m13.909188724s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.119177670 (232h8m13.922536847s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.123143490 (232h8m13.92650267s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.136750677 (232h8m13.940109854s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.348804845 (232h8m14.152164027s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.371058396 (232h8m14.17441757s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.387505238 (232h8m14.190864418s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.391295708 (232h8m14.194654919s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.542261626 (232h8m14.345620799s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.558338658 (232h8m14.36169784s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.559532458 (232h8m14.362891639s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.592708624 (232h8m14.396067801s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603936163 (232h8m14.407295329s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603990680 (232h8m14.407349863s) [seg_pos={0 0}, upload=241649]: Satisfied cohort requirements after 14 pieces
Feb 27 12:24:13.604300419 (232h8m14.407659603s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12J7goNH48bnMJyaCAPgUznrX1n6bNFNZXGqpuftNLVLxQxprsF): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604313894 (232h8m14.407673059s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604426170 (232h8m14.407785355s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XfehFoLvcivSkVcGttnXVK5VCSWKCpmyPX3E6rtpVdwnUbbZ4): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604545884 (232h8m14.407905074s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604898398 (232h8m14.408257582s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604908271 (232h8m14.408267455s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604924749 (232h8m14.408283934s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RQz653n137xXR4Dk2iXCDB1uKndLVm7C48VSa8B1J2tj2QvqR): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604936657 (232h8m14.408295846s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604988498 (232h8m14.408347659s) [seg_pos={0 0}, upload=241649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Feb 27 12:24:13.605271411 (232h8m14.408630597s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 27 12:24:13.605282821 (232h8m14.408642006s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 27 12:24:13.658039633 (232h8m14.461398807s) [upload=241649]: Done uploading segments...
Feb 27 12:24:13.658060724 (232h8m14.4614199s) [upload=241649]: Upload finished. err: <nil>
Duration: 1.237328808s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.867383ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Vary=Origin X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:41:41 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Amz-Request-Id=18981AF6F4E027D8 X-Request-Id=QiMqL8jQBFk target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/2ce03f87de3454be
FAIL (2026-02-27 12:41:41 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AF6F4E027D8</RequestId><HostId></HostId></Error>"
Duration: 10.906424ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
14.623614ms remote:
"tcp:136.0.77.3:443" response headers:
X-Amz-Request-Id=18981AF6E3EF0B6F Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content X-Request-Id=S9t9hRGFizg X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:41:41 GMT Content-Type=application/xml Server=Storj Vary=Origin target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/344994871245ef40
FAIL (2026-02-27 12:41:41 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AF6E3EF0B6F</RequestId><HostId></HostId></Error>"
Duration: 14.652089ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
13.025107ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Server=Storj Vary=Origin X-Request-Id=WnPpNJz9eHg X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:41:33 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Amz-Request-Id=18981AF53DFF39BC target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/c11dd28ffde77eb
FAIL (2026-02-27 12:41:33 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AF53DFF39BC</RequestId><HostId></HostId></Error>"
Duration: 13.07242ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
27.041776ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=NHJdJULzcUY Date=Fri, 27 Feb 2026 12:41:22 GMT Accept-Ranges=bytes X-Amz-Request-Id=18981AF2A67FD4FA X-Xss-Protection=1; mode=block Content-Length=284 Content-Security-Policy=block-all-mixed-content target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/7cbfb642da4293ee
FAIL (2026-02-27 12:41:22 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AF2A67FD4FA</RequestId><HostId></HostId></Error>"
Duration: 27.066632ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.29067ms remote:
"tcp:136.0.77.3:443" response headers:
Date=Fri, 27 Feb 2026 12:41:22 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Xss-Protection=1; mode=block Content-Length=274 Vary=Origin X-Amz-Request-Id=18981AF2876F553A X-Request-Id=AuDQAK8AYFN target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/4290b420e542e70
FAIL (2026-02-27 12:41:22 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AF2876F553A</RequestId><HostId></HostId></Error>"
Duration: 10.389584ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
1.355343286s trace:
https://jaeger.dev.storj.io/trace/41604b8e737e0644
FAIL (2026-02-27 12:41:08 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
Feb 27 12:24:12.958062237 (232h8m13.761421427s) [upload=241649]: Starting upload
Feb 27 12:24:12.958140898 (232h8m13.761500081s) [upload=241649]: Done starting upload
Feb 27 12:24:12.958361897 (232h8m13.761721063s) [upload=241649]: Uploading segments...
Feb 27 12:24:12.958461336 (232h8m13.761820517s) [upload=241649]: Got next segment. Inline: false
Feb 27 12:24:12.958521453 (232h8m13.761880637s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 27 12:24:12.958527047 (232h8m13.761886222s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 27 12:24:12.996321566 (232h8m13.79968075s) [seg_pos={0 0}, upload=241649]: Begin upload segment...
Feb 27 12:24:12.996448820 (232h8m13.799807999s) [seg_pos={0 0}, upload=241649]: Done begin upload segment.
Feb 27 12:24:12.996528741 (232h8m13.799887926s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.996657536 (232h8m13.800016719s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.998274501 (232h8m13.801633687s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999072752 (232h8m13.802431938s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999746928 (232h8m13.803106112s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000114348 (232h8m13.803473543s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000906074 (232h8m13.804265302s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000989520 (232h8m13.804348704s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001455955 (232h8m13.804815197s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001604735 (232h8m13.804963973s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002253164 (232h8m13.805612348s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002863992 (232h8m13.806223173s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003499182 (232h8m13.806858378s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003922169 (232h8m13.807281352s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004123035 (232h8m13.807482216s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004561506 (232h8m13.807920691s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.005503797 (232h8m13.808862985s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.006763318 (232h8m13.810122494s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.096582917 (232h8m13.899942093s) [upload=241649]: Next returned nil segment
Feb 27 12:24:13.096600442 (232h8m13.899959621s) [upload=241649]: Waiting for error group managing segments...
Feb 27 12:24:13.100656351 (232h8m13.904015632s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.105829543 (232h8m13.909188724s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.119177670 (232h8m13.922536847s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.123143490 (232h8m13.92650267s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.136750677 (232h8m13.940109854s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.348804845 (232h8m14.152164027s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.371058396 (232h8m14.17441757s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.387505238 (232h8m14.190864418s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.391295708 (232h8m14.194654919s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.542261626 (232h8m14.345620799s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.558338658 (232h8m14.36169784s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.559532458 (232h8m14.362891639s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.592708624 (232h8m14.396067801s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603936163 (232h8m14.407295329s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603990680 (232h8m14.407349863s) [seg_pos={0 0}, upload=241649]: Satisfied cohort requirements after 14 pieces
Feb 27 12:24:13.604300419 (232h8m14.407659603s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12J7goNH48bnMJyaCAPgUznrX1n6bNFNZXGqpuftNLVLxQxprsF): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604313894 (232h8m14.407673059s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604426170 (232h8m14.407785355s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XfehFoLvcivSkVcGttnXVK5VCSWKCpmyPX3E6rtpVdwnUbbZ4): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604545884 (232h8m14.407905074s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604898398 (232h8m14.408257582s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604908271 (232h8m14.408267455s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604924749 (232h8m14.408283934s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RQz653n137xXR4Dk2iXCDB1uKndLVm7C48VSa8B1J2tj2QvqR): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604936657 (232h8m14.408295846s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604988498 (232h8m14.408347659s) [seg_pos={0 0}, upload=241649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Feb 27 12:24:13.605271411 (232h8m14.408630597s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 27 12:24:13.605282821 (232h8m14.408642006s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 27 12:24:13.658039633 (232h8m14.461398807s) [upload=241649]: Done uploading segments...
Feb 27 12:24:13.658060724 (232h8m14.4614199s) [upload=241649]: Upload finished. err: <nil>
Duration: 1.355370557s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
31.429922ms remote:
"tcp:45.38.24.2:443" response headers:
Accept-Ranges=bytes Content-Length=284 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=UNaWTc6797w Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18981AEC89246518 X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:40:56 GMT target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/2eb390b7bc231215
FAIL (2026-02-27 12:40:56 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AEC89246518</RequestId><HostId></HostId></Error>"
Duration: 31.523572ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
16.074466ms remote:
"tcp:136.0.77.3:443" response headers:
Vary=Origin X-Request-Id=6bHg7XQBnTQ X-Xss-Protection=1; mode=block Content-Type=application/xml X-Amz-Request-Id=18981AEB6DDC8E1D Date=Fri, 27 Feb 2026 12:40:51 GMT Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Server=Storj target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/480822fdfb1f53e6
FAIL (2026-02-27 12:40:51 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AEB6DDC8E1D</RequestId><HostId></HostId></Error>"
Duration: 16.088944ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.148538ms remote:
"tcp:136.0.77.3:443" response headers:
Date=Fri, 27 Feb 2026 12:40:34 GMT Content-Length=274 Content-Type=application/xml X-Request-Id=coWijwMRcQ4 X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Amz-Request-Id=18981AE759C26FDE target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/d1f6927d66d2f49
FAIL (2026-02-27 12:40:34 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AE759C26FDE</RequestId><HostId></HostId></Error>"
Duration: 10.178152ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.007378ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Length=274 Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=18981AE51514CF4C X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:40:24 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content X-Request-Id=cf3GcTGAdUr target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/331b9c64e2993dbc
FAIL (2026-02-27 12:40:24 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AE51514CF4C</RequestId><HostId></HostId></Error>"
Duration: 11.11161ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.766613ms remote:
"tcp:136.0.77.3:443" response headers:
Server=Storj X-Request-Id=YxQTuKsbH6r Date=Fri, 27 Feb 2026 12:40:18 GMT Accept-Ranges=bytes Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18981AE38FCB99D9 X-Xss-Protection=1; mode=block Content-Length=274 Content-Security-Policy=block-all-mixed-content target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/37470c05008c0ff1
FAIL (2026-02-27 12:40:18 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AE38FCB99D9</RequestId><HostId></HostId></Error>"
Duration: 30.800326ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
1.295668592s trace:
https://jaeger.dev.storj.io/trace/41604b8e737e0644
FAIL (2026-02-27 12:40:12 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
Feb 27 12:24:12.958062237 (232h8m13.761421427s) [upload=241649]: Starting upload
Feb 27 12:24:12.958140898 (232h8m13.761500081s) [upload=241649]: Done starting upload
Feb 27 12:24:12.958361897 (232h8m13.761721063s) [upload=241649]: Uploading segments...
Feb 27 12:24:12.958461336 (232h8m13.761820517s) [upload=241649]: Got next segment. Inline: false
Feb 27 12:24:12.958521453 (232h8m13.761880637s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 27 12:24:12.958527047 (232h8m13.761886222s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 27 12:24:12.996321566 (232h8m13.79968075s) [seg_pos={0 0}, upload=241649]: Begin upload segment...
Feb 27 12:24:12.996448820 (232h8m13.799807999s) [seg_pos={0 0}, upload=241649]: Done begin upload segment.
Feb 27 12:24:12.996528741 (232h8m13.799887926s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.996657536 (232h8m13.800016719s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.998274501 (232h8m13.801633687s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999072752 (232h8m13.802431938s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999746928 (232h8m13.803106112s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000114348 (232h8m13.803473543s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000906074 (232h8m13.804265302s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000989520 (232h8m13.804348704s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001455955 (232h8m13.804815197s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001604735 (232h8m13.804963973s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002253164 (232h8m13.805612348s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002863992 (232h8m13.806223173s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003499182 (232h8m13.806858378s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003922169 (232h8m13.807281352s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004123035 (232h8m13.807482216s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004561506 (232h8m13.807920691s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.005503797 (232h8m13.808862985s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.006763318 (232h8m13.810122494s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.096582917 (232h8m13.899942093s) [upload=241649]: Next returned nil segment
Feb 27 12:24:13.096600442 (232h8m13.899959621s) [upload=241649]: Waiting for error group managing segments...
Feb 27 12:24:13.100656351 (232h8m13.904015632s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.105829543 (232h8m13.909188724s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.119177670 (232h8m13.922536847s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.123143490 (232h8m13.92650267s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.136750677 (232h8m13.940109854s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.348804845 (232h8m14.152164027s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.371058396 (232h8m14.17441757s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.387505238 (232h8m14.190864418s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.391295708 (232h8m14.194654919s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.542261626 (232h8m14.345620799s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.558338658 (232h8m14.36169784s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.559532458 (232h8m14.362891639s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.592708624 (232h8m14.396067801s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603936163 (232h8m14.407295329s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603990680 (232h8m14.407349863s) [seg_pos={0 0}, upload=241649]: Satisfied cohort requirements after 14 pieces
Feb 27 12:24:13.604300419 (232h8m14.407659603s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12J7goNH48bnMJyaCAPgUznrX1n6bNFNZXGqpuftNLVLxQxprsF): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604313894 (232h8m14.407673059s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604426170 (232h8m14.407785355s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XfehFoLvcivSkVcGttnXVK5VCSWKCpmyPX3E6rtpVdwnUbbZ4): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604545884 (232h8m14.407905074s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604898398 (232h8m14.408257582s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604908271 (232h8m14.408267455s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604924749 (232h8m14.408283934s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RQz653n137xXR4Dk2iXCDB1uKndLVm7C48VSa8B1J2tj2QvqR): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604936657 (232h8m14.408295846s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604988498 (232h8m14.408347659s) [seg_pos={0 0}, upload=241649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Feb 27 12:24:13.605271411 (232h8m14.408630597s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 27 12:24:13.605282821 (232h8m14.408642006s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 27 12:24:13.658039633 (232h8m14.461398807s) [upload=241649]: Done uploading segments...
Feb 27 12:24:13.658060724 (232h8m14.4614199s) [upload=241649]: Upload finished. err: <nil>
Duration: 1.295894831s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.680235ms remote:
"tcp:136.0.77.3:443" response headers:
Date=Fri, 27 Feb 2026 12:40:06 GMT Accept-Ranges=bytes Content-Length=274 X-Amz-Request-Id=18981AE0FC570A3B Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=LWSSjFYzScY X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/4a4ce8f5b85b6da5
FAIL (2026-02-27 12:40:06 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AE0FC570A3B</RequestId><HostId></HostId></Error>"
Duration: 11.710885ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
12.036455ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=Z9ZcGVKu64X X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:39:59 GMT Content-Length=274 X-Amz-Request-Id=18981ADF26DC2216 Accept-Ranges=bytes target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/5e24aa4d460b7cd0
FAIL (2026-02-27 12:39:59 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981ADF26DC2216</RequestId><HostId></HostId></Error>"
Duration: 12.065388ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
14.312378ms remote:
"tcp:136.0.77.3:443" response headers:
Date=Fri, 27 Feb 2026 12:39:53 GMT Accept-Ranges=bytes Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=18981ADDE9D5CB31 Content-Length=274 Content-Security-Policy=block-all-mixed-content X-Request-Id=Tc2TjJDQYAs X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/22b90c20ceeea7c1
FAIL (2026-02-27 12:39:53 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981ADDE9D5CB31</RequestId><HostId></HostId></Error>"
Duration: 14.35373ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
28.066453ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=18981ADC32BC30CC Accept-Ranges=bytes Content-Length=284 X-Request-Id=NdNtcXsVVLd X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:39:46 GMT target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/7955458a9c8525fe
FAIL (2026-02-27 12:39:46 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981ADC32BC30CC</RequestId><HostId></HostId></Error>"
Duration: 28.087107ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
1.244004718s trace:
https://jaeger.dev.storj.io/trace/41604b8e737e0644
FAIL (2026-02-27 12:39:26 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
Feb 27 12:24:12.958062237 (232h8m13.761421427s) [upload=241649]: Starting upload
Feb 27 12:24:12.958140898 (232h8m13.761500081s) [upload=241649]: Done starting upload
Feb 27 12:24:12.958361897 (232h8m13.761721063s) [upload=241649]: Uploading segments...
Feb 27 12:24:12.958461336 (232h8m13.761820517s) [upload=241649]: Got next segment. Inline: false
Feb 27 12:24:12.958521453 (232h8m13.761880637s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 27 12:24:12.958527047 (232h8m13.761886222s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 27 12:24:12.996321566 (232h8m13.79968075s) [seg_pos={0 0}, upload=241649]: Begin upload segment...
Feb 27 12:24:12.996448820 (232h8m13.799807999s) [seg_pos={0 0}, upload=241649]: Done begin upload segment.
Feb 27 12:24:12.996528741 (232h8m13.799887926s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.996657536 (232h8m13.800016719s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.998274501 (232h8m13.801633687s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999072752 (232h8m13.802431938s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999746928 (232h8m13.803106112s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000114348 (232h8m13.803473543s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000906074 (232h8m13.804265302s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000989520 (232h8m13.804348704s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001455955 (232h8m13.804815197s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001604735 (232h8m13.804963973s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002253164 (232h8m13.805612348s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002863992 (232h8m13.806223173s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003499182 (232h8m13.806858378s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003922169 (232h8m13.807281352s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004123035 (232h8m13.807482216s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004561506 (232h8m13.807920691s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.005503797 (232h8m13.808862985s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.006763318 (232h8m13.810122494s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.096582917 (232h8m13.899942093s) [upload=241649]: Next returned nil segment
Feb 27 12:24:13.096600442 (232h8m13.899959621s) [upload=241649]: Waiting for error group managing segments...
Feb 27 12:24:13.100656351 (232h8m13.904015632s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.105829543 (232h8m13.909188724s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.119177670 (232h8m13.922536847s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.123143490 (232h8m13.92650267s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.136750677 (232h8m13.940109854s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.348804845 (232h8m14.152164027s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.371058396 (232h8m14.17441757s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.387505238 (232h8m14.190864418s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.391295708 (232h8m14.194654919s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.542261626 (232h8m14.345620799s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.558338658 (232h8m14.36169784s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.559532458 (232h8m14.362891639s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.592708624 (232h8m14.396067801s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603936163 (232h8m14.407295329s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603990680 (232h8m14.407349863s) [seg_pos={0 0}, upload=241649]: Satisfied cohort requirements after 14 pieces
Feb 27 12:24:13.604300419 (232h8m14.407659603s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12J7goNH48bnMJyaCAPgUznrX1n6bNFNZXGqpuftNLVLxQxprsF): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604313894 (232h8m14.407673059s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604426170 (232h8m14.407785355s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XfehFoLvcivSkVcGttnXVK5VCSWKCpmyPX3E6rtpVdwnUbbZ4): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604545884 (232h8m14.407905074s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604898398 (232h8m14.408257582s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604908271 (232h8m14.408267455s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604924749 (232h8m14.408283934s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RQz653n137xXR4Dk2iXCDB1uKndLVm7C48VSa8B1J2tj2QvqR): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604936657 (232h8m14.408295846s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604988498 (232h8m14.408347659s) [seg_pos={0 0}, upload=241649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Feb 27 12:24:13.605271411 (232h8m14.408630597s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 27 12:24:13.605282821 (232h8m14.408642006s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 27 12:24:13.658039633 (232h8m14.461398807s) [upload=241649]: Done uploading segments...
Feb 27 12:24:13.658060724 (232h8m14.4614199s) [upload=241649]: Upload finished. err: <nil>
Duration: 1.244049343s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
12.599222ms remote:
"tcp:136.0.77.3:443" response headers:
Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:38:54 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Amz-Request-Id=18981AD036DC180B X-Request-Id=Hk6K3tJycvT Content-Length=274 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/f18967da525497a
FAIL (2026-02-27 12:38:54 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AD036DC180B</RequestId><HostId></HostId></Error>"
Duration: 12.689043ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
13.203636ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=18981ACF6DD5CAAA X-Request-Id=LiRDm9LCS9y X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Date=Fri, 27 Feb 2026 12:38:51 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/19798892a56e5548
FAIL (2026-02-27 12:38:51 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981ACF6DD5CAAA</RequestId><HostId></HostId></Error>"
Duration: 13.248875ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.103217ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Server=Storj X-Request-Id=DgA5EqR9VqG Date=Fri, 27 Feb 2026 12:38:51 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18981ACF80E4F121 X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/12103393715cbac6
FAIL (2026-02-27 12:38:51 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981ACF80E4F121</RequestId><HostId></HostId></Error>"
Duration: 11.132349ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
1.267891991s trace:
https://jaeger.dev.storj.io/trace/41604b8e737e0644
FAIL (2026-02-27 12:38:47 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
Feb 27 12:24:12.958062237 (232h8m13.761421427s) [upload=241649]: Starting upload
Feb 27 12:24:12.958140898 (232h8m13.761500081s) [upload=241649]: Done starting upload
Feb 27 12:24:12.958361897 (232h8m13.761721063s) [upload=241649]: Uploading segments...
Feb 27 12:24:12.958461336 (232h8m13.761820517s) [upload=241649]: Got next segment. Inline: false
Feb 27 12:24:12.958521453 (232h8m13.761880637s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 27 12:24:12.958527047 (232h8m13.761886222s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 27 12:24:12.996321566 (232h8m13.79968075s) [seg_pos={0 0}, upload=241649]: Begin upload segment...
Feb 27 12:24:12.996448820 (232h8m13.799807999s) [seg_pos={0 0}, upload=241649]: Done begin upload segment.
Feb 27 12:24:12.996528741 (232h8m13.799887926s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.996657536 (232h8m13.800016719s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.998274501 (232h8m13.801633687s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999072752 (232h8m13.802431938s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999746928 (232h8m13.803106112s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000114348 (232h8m13.803473543s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000906074 (232h8m13.804265302s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000989520 (232h8m13.804348704s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001455955 (232h8m13.804815197s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001604735 (232h8m13.804963973s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002253164 (232h8m13.805612348s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002863992 (232h8m13.806223173s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003499182 (232h8m13.806858378s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003922169 (232h8m13.807281352s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004123035 (232h8m13.807482216s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004561506 (232h8m13.807920691s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.005503797 (232h8m13.808862985s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.006763318 (232h8m13.810122494s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.096582917 (232h8m13.899942093s) [upload=241649]: Next returned nil segment
Feb 27 12:24:13.096600442 (232h8m13.899959621s) [upload=241649]: Waiting for error group managing segments...
Feb 27 12:24:13.100656351 (232h8m13.904015632s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.105829543 (232h8m13.909188724s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.119177670 (232h8m13.922536847s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.123143490 (232h8m13.92650267s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.136750677 (232h8m13.940109854s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.348804845 (232h8m14.152164027s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.371058396 (232h8m14.17441757s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.387505238 (232h8m14.190864418s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.391295708 (232h8m14.194654919s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.542261626 (232h8m14.345620799s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.558338658 (232h8m14.36169784s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.559532458 (232h8m14.362891639s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.592708624 (232h8m14.396067801s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603936163 (232h8m14.407295329s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603990680 (232h8m14.407349863s) [seg_pos={0 0}, upload=241649]: Satisfied cohort requirements after 14 pieces
Feb 27 12:24:13.604300419 (232h8m14.407659603s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12J7goNH48bnMJyaCAPgUznrX1n6bNFNZXGqpuftNLVLxQxprsF): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604313894 (232h8m14.407673059s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604426170 (232h8m14.407785355s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XfehFoLvcivSkVcGttnXVK5VCSWKCpmyPX3E6rtpVdwnUbbZ4): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604545884 (232h8m14.407905074s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604898398 (232h8m14.408257582s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604908271 (232h8m14.408267455s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604924749 (232h8m14.408283934s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RQz653n137xXR4Dk2iXCDB1uKndLVm7C48VSa8B1J2tj2QvqR): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604936657 (232h8m14.408295846s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604988498 (232h8m14.408347659s) [seg_pos={0 0}, upload=241649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Feb 27 12:24:13.605271411 (232h8m14.408630597s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 27 12:24:13.605282821 (232h8m14.408642006s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 27 12:24:13.658039633 (232h8m14.461398807s) [upload=241649]: Done uploading segments...
Feb 27 12:24:13.658060724 (232h8m14.4614199s) [upload=241649]: Upload finished. err: <nil>
Duration: 1.26792483s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
34.360875ms remote:
"tcp:45.38.24.2:443" response headers:
X-Amz-Request-Id=18981ACCDB4F88A7 Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Request-Id=JSFoG4erb5m X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:38:40 GMT Content-Length=284 Content-Type=application/xml target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/58747f28433ca197
FAIL (2026-02-27 12:38:40 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981ACCDB4F88A7</RequestId><HostId></HostId></Error>"
Duration: 34.404509ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
12.421469ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Type=application/xml X-Amz-Request-Id=18981AC9184DFAB9 X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:38:24 GMT Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Request-Id=5YdJPo6hmrC target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/355a1d8ef3635b63
FAIL (2026-02-27 12:38:24 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AC9184DFAB9</RequestId><HostId></HostId></Error>"
Duration: 12.466376ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
1.246702341s trace:
https://jaeger.dev.storj.io/trace/41604b8e737e0644
FAIL (2026-02-27 12:38:19 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
Feb 27 12:24:12.958062237 (232h8m13.761421427s) [upload=241649]: Starting upload
Feb 27 12:24:12.958140898 (232h8m13.761500081s) [upload=241649]: Done starting upload
Feb 27 12:24:12.958361897 (232h8m13.761721063s) [upload=241649]: Uploading segments...
Feb 27 12:24:12.958461336 (232h8m13.761820517s) [upload=241649]: Got next segment. Inline: false
Feb 27 12:24:12.958521453 (232h8m13.761880637s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 27 12:24:12.958527047 (232h8m13.761886222s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 27 12:24:12.996321566 (232h8m13.79968075s) [seg_pos={0 0}, upload=241649]: Begin upload segment...
Feb 27 12:24:12.996448820 (232h8m13.799807999s) [seg_pos={0 0}, upload=241649]: Done begin upload segment.
Feb 27 12:24:12.996528741 (232h8m13.799887926s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.996657536 (232h8m13.800016719s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.998274501 (232h8m13.801633687s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999072752 (232h8m13.802431938s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999746928 (232h8m13.803106112s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000114348 (232h8m13.803473543s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000906074 (232h8m13.804265302s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000989520 (232h8m13.804348704s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001455955 (232h8m13.804815197s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001604735 (232h8m13.804963973s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002253164 (232h8m13.805612348s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002863992 (232h8m13.806223173s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003499182 (232h8m13.806858378s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003922169 (232h8m13.807281352s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004123035 (232h8m13.807482216s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004561506 (232h8m13.807920691s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.005503797 (232h8m13.808862985s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.006763318 (232h8m13.810122494s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.096582917 (232h8m13.899942093s) [upload=241649]: Next returned nil segment
Feb 27 12:24:13.096600442 (232h8m13.899959621s) [upload=241649]: Waiting for error group managing segments...
Feb 27 12:24:13.100656351 (232h8m13.904015632s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.105829543 (232h8m13.909188724s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.119177670 (232h8m13.922536847s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.123143490 (232h8m13.92650267s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.136750677 (232h8m13.940109854s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.348804845 (232h8m14.152164027s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.371058396 (232h8m14.17441757s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.387505238 (232h8m14.190864418s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.391295708 (232h8m14.194654919s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.542261626 (232h8m14.345620799s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.558338658 (232h8m14.36169784s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.559532458 (232h8m14.362891639s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.592708624 (232h8m14.396067801s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603936163 (232h8m14.407295329s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603990680 (232h8m14.407349863s) [seg_pos={0 0}, upload=241649]: Satisfied cohort requirements after 14 pieces
Feb 27 12:24:13.604300419 (232h8m14.407659603s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12J7goNH48bnMJyaCAPgUznrX1n6bNFNZXGqpuftNLVLxQxprsF): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604313894 (232h8m14.407673059s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604426170 (232h8m14.407785355s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XfehFoLvcivSkVcGttnXVK5VCSWKCpmyPX3E6rtpVdwnUbbZ4): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604545884 (232h8m14.407905074s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604898398 (232h8m14.408257582s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604908271 (232h8m14.408267455s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604924749 (232h8m14.408283934s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RQz653n137xXR4Dk2iXCDB1uKndLVm7C48VSa8B1J2tj2QvqR): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604936657 (232h8m14.408295846s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604988498 (232h8m14.408347659s) [seg_pos={0 0}, upload=241649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Feb 27 12:24:13.605271411 (232h8m14.408630597s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 27 12:24:13.605282821 (232h8m14.408642006s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 27 12:24:13.658039633 (232h8m14.461398807s) [upload=241649]: Done uploading segments...
Feb 27 12:24:13.658060724 (232h8m14.4614199s) [upload=241649]: Upload finished. err: <nil>
Duration: 1.246729074s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.738309ms remote:
"tcp:136.0.77.3:443" response headers:
Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Amz-Request-Id=18981AC55A4E5D92 X-Request-Id=eB3b3MRR2wd Date=Fri, 27 Feb 2026 12:38:08 GMT Accept-Ranges=bytes Content-Length=274 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/618bd340e1e5a36f
FAIL (2026-02-27 12:38:08 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AC55A4E5D92</RequestId><HostId></HostId></Error>"
Duration: 9.774631ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
1.44134637s trace:
https://jaeger.dev.storj.io/trace/41604b8e737e0644
FAIL (2026-02-27 12:38:03 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
Feb 27 12:24:12.958062237 (232h8m13.761421427s) [upload=241649]: Starting upload
Feb 27 12:24:12.958140898 (232h8m13.761500081s) [upload=241649]: Done starting upload
Feb 27 12:24:12.958361897 (232h8m13.761721063s) [upload=241649]: Uploading segments...
Feb 27 12:24:12.958461336 (232h8m13.761820517s) [upload=241649]: Got next segment. Inline: false
Feb 27 12:24:12.958521453 (232h8m13.761880637s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 27 12:24:12.958527047 (232h8m13.761886222s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 27 12:24:12.996321566 (232h8m13.79968075s) [seg_pos={0 0}, upload=241649]: Begin upload segment...
Feb 27 12:24:12.996448820 (232h8m13.799807999s) [seg_pos={0 0}, upload=241649]: Done begin upload segment.
Feb 27 12:24:12.996528741 (232h8m13.799887926s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.996657536 (232h8m13.800016719s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.998274501 (232h8m13.801633687s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999072752 (232h8m13.802431938s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999746928 (232h8m13.803106112s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000114348 (232h8m13.803473543s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000906074 (232h8m13.804265302s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000989520 (232h8m13.804348704s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001455955 (232h8m13.804815197s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001604735 (232h8m13.804963973s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002253164 (232h8m13.805612348s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002863992 (232h8m13.806223173s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003499182 (232h8m13.806858378s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003922169 (232h8m13.807281352s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004123035 (232h8m13.807482216s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004561506 (232h8m13.807920691s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.005503797 (232h8m13.808862985s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.006763318 (232h8m13.810122494s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.096582917 (232h8m13.899942093s) [upload=241649]: Next returned nil segment
Feb 27 12:24:13.096600442 (232h8m13.899959621s) [upload=241649]: Waiting for error group managing segments...
Feb 27 12:24:13.100656351 (232h8m13.904015632s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.105829543 (232h8m13.909188724s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.119177670 (232h8m13.922536847s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.123143490 (232h8m13.92650267s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.136750677 (232h8m13.940109854s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.348804845 (232h8m14.152164027s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.371058396 (232h8m14.17441757s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.387505238 (232h8m14.190864418s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.391295708 (232h8m14.194654919s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.542261626 (232h8m14.345620799s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.558338658 (232h8m14.36169784s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.559532458 (232h8m14.362891639s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.592708624 (232h8m14.396067801s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603936163 (232h8m14.407295329s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603990680 (232h8m14.407349863s) [seg_pos={0 0}, upload=241649]: Satisfied cohort requirements after 14 pieces
Feb 27 12:24:13.604300419 (232h8m14.407659603s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12J7goNH48bnMJyaCAPgUznrX1n6bNFNZXGqpuftNLVLxQxprsF): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604313894 (232h8m14.407673059s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604426170 (232h8m14.407785355s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XfehFoLvcivSkVcGttnXVK5VCSWKCpmyPX3E6rtpVdwnUbbZ4): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604545884 (232h8m14.407905074s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604898398 (232h8m14.408257582s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604908271 (232h8m14.408267455s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604924749 (232h8m14.408283934s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RQz653n137xXR4Dk2iXCDB1uKndLVm7C48VSa8B1J2tj2QvqR): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604936657 (232h8m14.408295846s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604988498 (232h8m14.408347659s) [seg_pos={0 0}, upload=241649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Feb 27 12:24:13.605271411 (232h8m14.408630597s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 27 12:24:13.605282821 (232h8m14.408642006s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 27 12:24:13.658039633 (232h8m14.461398807s) [upload=241649]: Done uploading segments...
Feb 27 12:24:13.658060724 (232h8m14.4614199s) [upload=241649]: Upload finished. err: <nil>
Duration: 1.441383149s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
12.975433ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Server=Storj Date=Fri, 27 Feb 2026 12:38:03 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18981AC43E6B0B36 X-Request-Id=ZFKyHm7Poaw X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/741327bee411a5b4
FAIL (2026-02-27 12:38:03 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AC43E6B0B36</RequestId><HostId></HostId></Error>"
Duration: 13.090874ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
27.978984ms remote:
"tcp:45.38.24.2:443" response headers:
X-Request-Id=eC262StvLov X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Server=Storj Date=Fri, 27 Feb 2026 12:37:51 GMT Content-Length=284 Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18981AC166B55960 target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/4dd413a722f041b8
FAIL (2026-02-27 12:37:51 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AC166B55960</RequestId><HostId></HostId></Error>"
Duration: 28.004952ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
1.127827575s trace:
https://jaeger.dev.storj.io/trace/41604b8e737e0644
FAIL (2026-02-27 12:37:46 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
Feb 27 12:24:12.958062237 (232h8m13.761421427s) [upload=241649]: Starting upload
Feb 27 12:24:12.958140898 (232h8m13.761500081s) [upload=241649]: Done starting upload
Feb 27 12:24:12.958361897 (232h8m13.761721063s) [upload=241649]: Uploading segments...
Feb 27 12:24:12.958461336 (232h8m13.761820517s) [upload=241649]: Got next segment. Inline: false
Feb 27 12:24:12.958521453 (232h8m13.761880637s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 27 12:24:12.958527047 (232h8m13.761886222s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 27 12:24:12.996321566 (232h8m13.79968075s) [seg_pos={0 0}, upload=241649]: Begin upload segment...
Feb 27 12:24:12.996448820 (232h8m13.799807999s) [seg_pos={0 0}, upload=241649]: Done begin upload segment.
Feb 27 12:24:12.996528741 (232h8m13.799887926s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.996657536 (232h8m13.800016719s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.998274501 (232h8m13.801633687s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999072752 (232h8m13.802431938s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999746928 (232h8m13.803106112s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000114348 (232h8m13.803473543s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000906074 (232h8m13.804265302s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000989520 (232h8m13.804348704s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001455955 (232h8m13.804815197s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001604735 (232h8m13.804963973s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002253164 (232h8m13.805612348s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002863992 (232h8m13.806223173s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003499182 (232h8m13.806858378s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003922169 (232h8m13.807281352s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004123035 (232h8m13.807482216s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004561506 (232h8m13.807920691s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.005503797 (232h8m13.808862985s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.006763318 (232h8m13.810122494s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.096582917 (232h8m13.899942093s) [upload=241649]: Next returned nil segment
Feb 27 12:24:13.096600442 (232h8m13.899959621s) [upload=241649]: Waiting for error group managing segments...
Feb 27 12:24:13.100656351 (232h8m13.904015632s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.105829543 (232h8m13.909188724s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.119177670 (232h8m13.922536847s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.123143490 (232h8m13.92650267s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.136750677 (232h8m13.940109854s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.348804845 (232h8m14.152164027s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.371058396 (232h8m14.17441757s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.387505238 (232h8m14.190864418s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.391295708 (232h8m14.194654919s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.542261626 (232h8m14.345620799s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.558338658 (232h8m14.36169784s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.559532458 (232h8m14.362891639s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.592708624 (232h8m14.396067801s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603936163 (232h8m14.407295329s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603990680 (232h8m14.407349863s) [seg_pos={0 0}, upload=241649]: Satisfied cohort requirements after 14 pieces
Feb 27 12:24:13.604300419 (232h8m14.407659603s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12J7goNH48bnMJyaCAPgUznrX1n6bNFNZXGqpuftNLVLxQxprsF): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604313894 (232h8m14.407673059s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604426170 (232h8m14.407785355s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XfehFoLvcivSkVcGttnXVK5VCSWKCpmyPX3E6rtpVdwnUbbZ4): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604545884 (232h8m14.407905074s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604898398 (232h8m14.408257582s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604908271 (232h8m14.408267455s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604924749 (232h8m14.408283934s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RQz653n137xXR4Dk2iXCDB1uKndLVm7C48VSa8B1J2tj2QvqR): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604936657 (232h8m14.408295846s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604988498 (232h8m14.408347659s) [seg_pos={0 0}, upload=241649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Feb 27 12:24:13.605271411 (232h8m14.408630597s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 27 12:24:13.605282821 (232h8m14.408642006s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 27 12:24:13.658039633 (232h8m14.461398807s) [upload=241649]: Done uploading segments...
Feb 27 12:24:13.658060724 (232h8m14.4614199s) [upload=241649]: Upload finished. err: <nil>
Duration: 1.127966132s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
12.165704ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:37:33 GMT Content-Length=274 Server=Storj Vary=Origin X-Amz-Request-Id=18981ABD5D526A78 X-Request-Id=jTBCtuGHswr target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/7dd70e27e5440301
FAIL (2026-02-27 12:37:33 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981ABD5D526A78</RequestId><HostId></HostId></Error>"
Duration: 12.194854ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
32.903651ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Type=application/xml X-Amz-Request-Id=18981ABB444630E2 X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:37:24 GMT Accept-Ranges=bytes Content-Length=284 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Request-Id=8yLzPqzYDaM target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/3b2f3064df263fec
FAIL (2026-02-27 12:37:24 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981ABB444630E2</RequestId><HostId></HostId></Error>"
Duration: 32.927319ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
1.229832673s trace:
https://jaeger.dev.storj.io/trace/41604b8e737e0644
FAIL (2026-02-27 12:37:22 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
Feb 27 12:24:12.958062237 (232h8m13.761421427s) [upload=241649]: Starting upload
Feb 27 12:24:12.958140898 (232h8m13.761500081s) [upload=241649]: Done starting upload
Feb 27 12:24:12.958361897 (232h8m13.761721063s) [upload=241649]: Uploading segments...
Feb 27 12:24:12.958461336 (232h8m13.761820517s) [upload=241649]: Got next segment. Inline: false
Feb 27 12:24:12.958521453 (232h8m13.761880637s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 27 12:24:12.958527047 (232h8m13.761886222s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 27 12:24:12.996321566 (232h8m13.79968075s) [seg_pos={0 0}, upload=241649]: Begin upload segment...
Feb 27 12:24:12.996448820 (232h8m13.799807999s) [seg_pos={0 0}, upload=241649]: Done begin upload segment.
Feb 27 12:24:12.996528741 (232h8m13.799887926s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.996657536 (232h8m13.800016719s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.998274501 (232h8m13.801633687s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999072752 (232h8m13.802431938s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999746928 (232h8m13.803106112s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000114348 (232h8m13.803473543s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000906074 (232h8m13.804265302s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000989520 (232h8m13.804348704s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001455955 (232h8m13.804815197s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001604735 (232h8m13.804963973s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002253164 (232h8m13.805612348s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002863992 (232h8m13.806223173s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003499182 (232h8m13.806858378s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003922169 (232h8m13.807281352s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004123035 (232h8m13.807482216s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004561506 (232h8m13.807920691s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.005503797 (232h8m13.808862985s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.006763318 (232h8m13.810122494s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.096582917 (232h8m13.899942093s) [upload=241649]: Next returned nil segment
Feb 27 12:24:13.096600442 (232h8m13.899959621s) [upload=241649]: Waiting for error group managing segments...
Feb 27 12:24:13.100656351 (232h8m13.904015632s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.105829543 (232h8m13.909188724s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.119177670 (232h8m13.922536847s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.123143490 (232h8m13.92650267s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.136750677 (232h8m13.940109854s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.348804845 (232h8m14.152164027s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.371058396 (232h8m14.17441757s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.387505238 (232h8m14.190864418s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.391295708 (232h8m14.194654919s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.542261626 (232h8m14.345620799s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.558338658 (232h8m14.36169784s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.559532458 (232h8m14.362891639s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.592708624 (232h8m14.396067801s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603936163 (232h8m14.407295329s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603990680 (232h8m14.407349863s) [seg_pos={0 0}, upload=241649]: Satisfied cohort requirements after 14 pieces
Feb 27 12:24:13.604300419 (232h8m14.407659603s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12J7goNH48bnMJyaCAPgUznrX1n6bNFNZXGqpuftNLVLxQxprsF): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604313894 (232h8m14.407673059s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604426170 (232h8m14.407785355s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XfehFoLvcivSkVcGttnXVK5VCSWKCpmyPX3E6rtpVdwnUbbZ4): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604545884 (232h8m14.407905074s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604898398 (232h8m14.408257582s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604908271 (232h8m14.408267455s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604924749 (232h8m14.408283934s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RQz653n137xXR4Dk2iXCDB1uKndLVm7C48VSa8B1J2tj2QvqR): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604936657 (232h8m14.408295846s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604988498 (232h8m14.408347659s) [seg_pos={0 0}, upload=241649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Feb 27 12:24:13.605271411 (232h8m14.408630597s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 27 12:24:13.605282821 (232h8m14.408642006s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 27 12:24:13.658039633 (232h8m14.461398807s) [upload=241649]: Done uploading segments...
Feb 27 12:24:13.658060724 (232h8m14.4614199s) [upload=241649]: Upload finished. err: <nil>
Duration: 1.229868108s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.547876ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=QGjNX8vqeUE X-Xss-Protection=1; mode=block Content-Length=274 X-Amz-Request-Id=18981ABA30DB2C2B Date=Fri, 27 Feb 2026 12:37:20 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/1877b16412fcfcb6
FAIL (2026-02-27 12:37:20 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981ABA30DB2C2B</RequestId><HostId></HostId></Error>"
Duration: 10.573483ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.33357ms remote:
"tcp:136.0.77.3:443" response headers:
X-Amz-Request-Id=18981AB903789B0D Date=Fri, 27 Feb 2026 12:37:15 GMT Content-Type=application/xml X-Request-Id=URjH2tx23ED X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/1829a5c8975c4f3b
FAIL (2026-02-27 12:37:15 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AB903789B0D</RequestId><HostId></HostId></Error>"
Duration: 10.421316ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
16.108416ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Amz-Request-Id=18981AB78C154D1B X-Request-Id=iq2Q2BoqyPC Date=Fri, 27 Feb 2026 12:37:09 GMT Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/6e29c7fb6db17281
FAIL (2026-02-27 12:37:09 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AB78C154D1B</RequestId><HostId></HostId></Error>"
Duration: 16.146689ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
1.242827216s trace:
https://jaeger.dev.storj.io/trace/41604b8e737e0644
FAIL (2026-02-27 12:36:52 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
Feb 27 12:24:12.958062237 (232h8m13.761421427s) [upload=241649]: Starting upload
Feb 27 12:24:12.958140898 (232h8m13.761500081s) [upload=241649]: Done starting upload
Feb 27 12:24:12.958361897 (232h8m13.761721063s) [upload=241649]: Uploading segments...
Feb 27 12:24:12.958461336 (232h8m13.761820517s) [upload=241649]: Got next segment. Inline: false
Feb 27 12:24:12.958521453 (232h8m13.761880637s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 27 12:24:12.958527047 (232h8m13.761886222s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 27 12:24:12.996321566 (232h8m13.79968075s) [seg_pos={0 0}, upload=241649]: Begin upload segment...
Feb 27 12:24:12.996448820 (232h8m13.799807999s) [seg_pos={0 0}, upload=241649]: Done begin upload segment.
Feb 27 12:24:12.996528741 (232h8m13.799887926s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.996657536 (232h8m13.800016719s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.998274501 (232h8m13.801633687s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999072752 (232h8m13.802431938s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999746928 (232h8m13.803106112s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000114348 (232h8m13.803473543s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000906074 (232h8m13.804265302s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000989520 (232h8m13.804348704s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001455955 (232h8m13.804815197s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001604735 (232h8m13.804963973s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002253164 (232h8m13.805612348s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002863992 (232h8m13.806223173s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003499182 (232h8m13.806858378s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003922169 (232h8m13.807281352s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004123035 (232h8m13.807482216s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004561506 (232h8m13.807920691s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.005503797 (232h8m13.808862985s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.006763318 (232h8m13.810122494s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.096582917 (232h8m13.899942093s) [upload=241649]: Next returned nil segment
Feb 27 12:24:13.096600442 (232h8m13.899959621s) [upload=241649]: Waiting for error group managing segments...
Feb 27 12:24:13.100656351 (232h8m13.904015632s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.105829543 (232h8m13.909188724s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.119177670 (232h8m13.922536847s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.123143490 (232h8m13.92650267s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.136750677 (232h8m13.940109854s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.348804845 (232h8m14.152164027s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.371058396 (232h8m14.17441757s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.387505238 (232h8m14.190864418s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.391295708 (232h8m14.194654919s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.542261626 (232h8m14.345620799s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.558338658 (232h8m14.36169784s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.559532458 (232h8m14.362891639s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.592708624 (232h8m14.396067801s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603936163 (232h8m14.407295329s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603990680 (232h8m14.407349863s) [seg_pos={0 0}, upload=241649]: Satisfied cohort requirements after 14 pieces
Feb 27 12:24:13.604300419 (232h8m14.407659603s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12J7goNH48bnMJyaCAPgUznrX1n6bNFNZXGqpuftNLVLxQxprsF): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604313894 (232h8m14.407673059s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604426170 (232h8m14.407785355s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XfehFoLvcivSkVcGttnXVK5VCSWKCpmyPX3E6rtpVdwnUbbZ4): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604545884 (232h8m14.407905074s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604898398 (232h8m14.408257582s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604908271 (232h8m14.408267455s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604924749 (232h8m14.408283934s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RQz653n137xXR4Dk2iXCDB1uKndLVm7C48VSa8B1J2tj2QvqR): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604936657 (232h8m14.408295846s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604988498 (232h8m14.408347659s) [seg_pos={0 0}, upload=241649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Feb 27 12:24:13.605271411 (232h8m14.408630597s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 27 12:24:13.605282821 (232h8m14.408642006s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 27 12:24:13.658039633 (232h8m14.461398807s) [upload=241649]: Done uploading segments...
Feb 27 12:24:13.658060724 (232h8m14.4614199s) [upload=241649]: Upload finished. err: <nil>
Duration: 1.242856538s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.583522ms remote:
"tcp:136.0.77.3:443" response headers:
X-Xss-Protection=1; mode=block Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18981AB3076A876B X-Request-Id=LyhiVfTBnp4 Date=Fri, 27 Feb 2026 12:36:49 GMT Accept-Ranges=bytes Content-Length=274 Server=Storj target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/52b64be3624273cb
FAIL (2026-02-27 12:36:49 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AB3076A876B</RequestId><HostId></HostId></Error>"
Duration: 10.608069ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
29.783694ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=JNCbL3Vg6XZ X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:36:33 GMT Accept-Ranges=bytes Content-Length=284 X-Amz-Request-Id=18981AAF3BDBD49D Content-Security-Policy=block-all-mixed-content target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/1411243b71ce2b72
FAIL (2026-02-27 12:36:33 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AAF3BDBD49D</RequestId><HostId></HostId></Error>"
Duration: 29.820605ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.991214ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=18981AAEC81A63FB X-Request-Id=SjWhDZq3Zva Content-Length=274 X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:36:31 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/1b7a793aa5dfc5f4
FAIL (2026-02-27 12:36:31 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AAEC81A63FB</RequestId><HostId></HostId></Error>"
Duration: 10.018449ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
13.509556ms remote:
"tcp:136.0.77.3:443" response headers:
X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:36:09 GMT Accept-Ranges=bytes Vary=Origin X-Amz-Request-Id=18981AA99FE8ECD6 X-Request-Id=MHCQdZDz16f Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/5a95718e1268bfb7
FAIL (2026-02-27 12:36:09 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AA99FE8ECD6</RequestId><HostId></HostId></Error>"
Duration: 13.553611ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.013963ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Date=Fri, 27 Feb 2026 12:36:09 GMT Server=Storj Vary=Origin X-Amz-Request-Id=18981AA9AE1A7CD2 X-Request-Id=XKWDusVzqE8 X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/688e73d372f1ecca
FAIL (2026-02-27 12:36:09 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AA9AE1A7CD2</RequestId><HostId></HostId></Error>"
Duration: 11.038752ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
1.263162472s trace:
https://jaeger.dev.storj.io/trace/41604b8e737e0644
FAIL (2026-02-27 12:35:50 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
Feb 27 12:24:12.958062237 (232h8m13.761421427s) [upload=241649]: Starting upload
Feb 27 12:24:12.958140898 (232h8m13.761500081s) [upload=241649]: Done starting upload
Feb 27 12:24:12.958361897 (232h8m13.761721063s) [upload=241649]: Uploading segments...
Feb 27 12:24:12.958461336 (232h8m13.761820517s) [upload=241649]: Got next segment. Inline: false
Feb 27 12:24:12.958521453 (232h8m13.761880637s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 27 12:24:12.958527047 (232h8m13.761886222s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 27 12:24:12.996321566 (232h8m13.79968075s) [seg_pos={0 0}, upload=241649]: Begin upload segment...
Feb 27 12:24:12.996448820 (232h8m13.799807999s) [seg_pos={0 0}, upload=241649]: Done begin upload segment.
Feb 27 12:24:12.996528741 (232h8m13.799887926s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.996657536 (232h8m13.800016719s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.998274501 (232h8m13.801633687s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999072752 (232h8m13.802431938s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:12.999746928 (232h8m13.803106112s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000114348 (232h8m13.803473543s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000906074 (232h8m13.804265302s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.000989520 (232h8m13.804348704s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001455955 (232h8m13.804815197s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.001604735 (232h8m13.804963973s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002253164 (232h8m13.805612348s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.002863992 (232h8m13.806223173s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003499182 (232h8m13.806858378s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.003922169 (232h8m13.807281352s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004123035 (232h8m13.807482216s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.004561506 (232h8m13.807920691s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.005503797 (232h8m13.808862985s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.006763318 (232h8m13.810122494s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Uploading piece...
Feb 27 12:24:13.096582917 (232h8m13.899942093s) [upload=241649]: Next returned nil segment
Feb 27 12:24:13.096600442 (232h8m13.899959621s) [upload=241649]: Waiting for error group managing segments...
Feb 27 12:24:13.100656351 (232h8m13.904015632s) [noise=true , address=77.90.31.17:4781 , piece_id=5WPZTO3KADH7PJRDVZHZRHFGPKCRGDUQFFSU2IASGA34WGTKITYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.105829543 (232h8m13.909188724s) [noise=true , address=145.79.227.17:4003 , piece_id=EOOSDAIIZ7SCDQYF6YM7X4KM3IBJ5ZZEUUC6NI7KP5U4SFTYVIGQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.119177670 (232h8m13.922536847s) [noise=true , address=45.91.116.8:4062 , piece_id=OOI4LTYZUSR4XIEWUQUE4I6TKMYCYVB3C4EZQMX3QZXIZ6VP3FRQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.123143490 (232h8m13.92650267s) [noise=true , address=77.90.31.16:4767 , piece_id=HB3F3CCZNCGUVV7GQCSCFKA3O65OD2C675RQ2GBEY7F455UOXX6A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.136750677 (232h8m13.940109854s) [noise=true , address=77.90.31.11:4557 , piece_id=6RGA2LKJU2GPDHZO5ZJAP5JV7FPJHPBP6FGEVUFPRO2DQP5QRRYA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.348804845 (232h8m14.152164027s) [noise=true , address=167.17.72.68:4650 , piece_id=YGUZCA6UR763GM6LLOBW63I3XV5QRLCDPNVPNBVI7CU4PTPJBL5Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.371058396 (232h8m14.17441757s) [noise=true , address=167.17.72.71:4781 , piece_id=4SOK636CRCLWNTDOBZLD6N2K3OG4QAQLORSXN4OYSHYVVCNOOY2A, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.387505238 (232h8m14.190864418s) [noise=true , address=167.17.72.69:4713 , piece_id=XOV243LRSIJ3AO37MWITSBYT56UCERPUAMOVKJYWTVHKQYPTGQBA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.391295708 (232h8m14.194654919s) [noise=true , address=167.17.72.70:4774 , piece_id=WOW24DU7IHJQDUJ7XWUBRRJJBQQAC6PNMAVMIHXSER5AHLQYLPIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.542261626 (232h8m14.345620799s) [noise=true , address=79.127.205.234:4198 , piece_id=P4UI3EJI4CU2ABQVNDKNFKVB6L2EWQYLBXTNHZYFTMWSZP655XLA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.558338658 (232h8m14.36169784s) [noise=true , address=79.127.205.233:4147 , piece_id=W5LZWDTG7UOBNTBMCTVYRI5YMGJXUEYENFY7TKMFIZHCYYXEGNIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.559532458 (232h8m14.362891639s) [noise=true , address=79.127.205.235:4252 , piece_id=4EGUQPE6WXCMZGDWNLLADBI47XVXWMP7323QAWZZZBD2FKVNRAPQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.592708624 (232h8m14.396067801s) [noise=true , address=193.149.164.67:4025 , piece_id=4E74ORPXOWTXYVX77P7JW5ECONNJY56LPH5J6OUZRH2EM2GZFB4Q, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603936163 (232h8m14.407295329s) [noise=true , address=193.149.164.69:4086 , piece_id=AYY5FI42QZRCZHRNP54G4F2X3U5PQ4ORAZZFYC2BV5T2MEDUSRVA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: <nil>
Feb 27 12:24:13.603990680 (232h8m14.407349863s) [seg_pos={0 0}, upload=241649]: Satisfied cohort requirements after 14 pieces
Feb 27 12:24:13.604300419 (232h8m14.407659603s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12J7goNH48bnMJyaCAPgUznrX1n6bNFNZXGqpuftNLVLxQxprsF): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604313894 (232h8m14.407673059s) [noise=true , address=145.79.227.43:4187 , piece_id=MDSPZEEVTTDM624JP7RVUT7TT3V7QQLIVZ2IAJNLPSEVSZGC5BIQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604426170 (232h8m14.407785355s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XfehFoLvcivSkVcGttnXVK5VCSWKCpmyPX3E6rtpVdwnUbbZ4): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604545884 (232h8m14.407905074s) [noise=true , address=193.149.164.68:4041 , piece_id=2ONFEAV67QPMJYNRD5DLK4WVHDELCTTVXOVX5U7D5PT4INGN6XIA, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604898398 (232h8m14.408257582s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604908271 (232h8m14.408267455s) [noise=true , address=145.79.227.38:4984 , piece_id=FQVQGMFTD42KOZWHKRVPT5A4XMXPTYGZ6BNK2ETVST3ZKBM4P3HQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604924749 (232h8m14.408283934s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RQz653n137xXR4Dk2iXCDB1uKndLVm7C48VSa8B1J2tj2QvqR): protocol: expected piece hash; context canceled; EOF
Feb 27 12:24:13.604936657 (232h8m14.408295846s) [noise=true , address=193.149.164.66:4963 , piece_id=5XKCRDLGMGAA6GBVVPXXY2K7HH4N5TRM2KAFBO3K5AFCMLNW53AQ, seg_pos={0 0}, upload=241649]: Long tail cancellation due to optimal threshold
Feb 27 12:24:13.604988498 (232h8m14.408347659s) [seg_pos={0 0}, upload=241649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Feb 27 12:24:13.605271411 (232h8m14.408630597s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 27 12:24:13.605282821 (232h8m14.408642006s) [upload=241649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 27 12:24:13.658039633 (232h8m14.461398807s) [upload=241649]: Done uploading segments...
Feb 27 12:24:13.658060724 (232h8m14.4614199s) [upload=241649]: Upload finished. err: <nil>
Duration: 1.263216679s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.614064ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Length=274 Vary=Origin X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:35:29 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Amz-Request-Id=18981AA07AFCD856 X-Request-Id=iwtidc2PxJx target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/6e3acd24566b79d9
FAIL (2026-02-27 12:35:29 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981AA07AFCD856</RequestId><HostId></HostId></Error>"
Duration: 11.725958ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.94059ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=18981A9FAE426B99 X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 12:35:26 GMT Content-Length=274 Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=8LQQT4uQtUA target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/588a6a44463004ca
FAIL (2026-02-27 12:35:26 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18981A9FAE426B99</RequestId><HostId></HostId></Error>"
Duration: 9.969681ms
edge
gateway
vhost
bgp
eu1
|