Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
14.60323ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Security-Policy=block-all-mixed-content Vary=Origin X-Request-Id=5b5oNGhZh7z X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:33:55 GMT Content-Length=274 Content-Type=application/xml Server=Storj X-Amz-Request-Id=18982E2FAB58AF21 Accept-Ranges=bytes target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/6b07e79b258f99da
FAIL (2026-02-27 18:33:55 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982E2FAB58AF21</RequestId><HostId></HostId></Error>"
Duration: 14.623675ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.453305ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18982E2D59BD3626 X-Request-Id=8QoinsNWKM9 X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:33:45 GMT Accept-Ranges=bytes Content-Length=274 Server=Storj target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/7dd48fed69cd9f95
FAIL (2026-02-27 18:33:45 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982E2D59BD3626</RequestId><HostId></HostId></Error>"
Duration: 11.542072ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [........................X......................X............] duration:
2.126095762s trace:
https://jaeger.dev.storj.io/trace/7aa57c119569a500
FAIL (2026-02-27 18:33:42 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
KCFQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408102653 (238h9m9.211461838s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408567070 (238h9m9.211926247s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408707192 (238h9m9.212066378s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408968070 (238h9m9.212327244s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409367498 (238h9m9.212726677s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409377018 (238h9m9.212736185s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410004538 (238h9m9.213363725s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410656413 (238h9m9.214015608s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411331444 (238h9m9.214690672s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411885321 (238h9m9.215244506s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412446978 (238h9m9.21580616s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412960206 (238h9m9.216319389s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.833976270 (238h9m9.637335445s) [upload=247902]: Next returned nil segment
Feb 27 18:25:08.833993781 (238h9m9.637352963s) [upload=247902]: Waiting for error group managing segments...
Feb 27 18:25:09.002997616 (238h9m9.806356791s) [noise=true , address=81.171.14.70:4040 , piece_id=2MPY2M6YD4A35CALBOQBGARWAVYBWSALZU2P4JQNVTVU462KXOSQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.003988167 (238h9m9.807347334s) [noise=true , address=81.171.14.70:4037 , piece_id=O6CQWKOJPHOHZAO7PMTX74XFWNPWIDDDUE2S4NPO67A64TRGVLDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.007807120 (238h9m9.811166324s) [noise=true , address=81.171.14.70:4044 , piece_id=SX5RETWJXQ37MNWETTT3ISLFDEQMJ4LGWEWLA4SELKUG6L447YZQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.019462938 (238h9m9.822822115s) [noise=true , address=81.171.14.70:4039 , piece_id=VV7S6QSQSAMDYP2J4TWHDJNIGMTPEHG7N4737DETH4SSYKK3FKNA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.031013955 (238h9m9.834373141s) [noise=true , address=185.167.204.206:4050 , piece_id=NNXAXJUAW34ZSFMQZHSMONU2KSGJ4DNNNLGA3VCV7Q7JE4OXYUAA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.039442786 (238h9m9.84280196s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.050960211 (238h9m9.854319391s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.067324682 (238h9m9.870683857s) [noise=true , address=185.167.204.206:4067 , piece_id=LMTR6XVP77QRHIFJFPK2CCNTLFVKGCNFXBN2Z4AQQTE7YTJ2PDVA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.070489496 (238h9m9.87384866s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.071657801 (238h9m9.87501698s) [noise=true , address=185.167.204.206:4068 , piece_id=DRL3HHGWZQGLK5RWPTJFIOUERAFO3OH3FWGWCVCPLN3JAR7K7QPA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.072680314 (238h9m9.876039488s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.074685847 (238h9m9.878045033s) [noise=true , address=185.167.204.205:4008 , piece_id=CDKH6EYARWHKZIMNICX46Z3JSGBETOJEPGVGGESJ7LZZYGGBKKMQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.089811558 (238h9m9.893170735s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.098264180 (238h9m9.901623357s) [noise=true , address=185.167.204.206:4053 , piece_id=GOQ52CA2J4GSR7A5DXDDDMEDVHODA4FE5A6A2I3GMDSICFLSKCFQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101019345 (238h9m9.904378509s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101369094 (238h9m9.904728292s) [noise=true , address=185.167.204.205:4002 , piece_id=2DSKXMEVZMV5ECFAXVPHHSK3YS5LQI3LQNTGYD3DNCHKT74VGRSA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.102365507 (238h9m9.905724671s) [noise=true , address=185.167.204.206:4041 , piece_id=JYD734S7X4QLARQFX44G24S7YXDUX3GWJ4QOY5NHD2NY5XMLVVOA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.103504275 (238h9m9.90686345s) [noise=true , address=89.149.201.147:4008 , piece_id=HY4JE6TBQ3AAXVGQNXVKIMX6MNU45UMLPK2YEH4QXC4OBKPNEWNQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.106507437 (238h9m9.909866612s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.145089732 (238h9m9.948448907s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.473005969 (238h9m10.276365143s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:13.403656159 (238h9m14.207015334s) [noise=true , address=185.167.204.206:4059 , piece_id=RYP2J5M7ERW64IPIDS4YC6XSMXJJYFY2FIOHFFVJ4FMEZMFC45PQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:14.229596272 (238h9m15.032955449s) [noise=true , address=185.167.204.206:4071 , piece_id=S23HGNGGEWNCQP2TMJ3CH2HL4TQWMEYVDFHCYKFAHXBDAY4X6ILA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:17.197218248 (238h9m18.000577422s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:50.077929419 (238h9m50.881288595s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:26:08.034087443 (238h10m8.837446626s) [noise=true , address=89.149.201.147:4016 , piece_id=DRJK3ZPIZKZ44E6RPX6FGQZWPNMA7VMAIP6JXCB7IUQQ7QC5F7BQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034276836 (238h10m8.837636013s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 159.223.143.39:60508->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034741100 (238h10m8.83810028s) [noise=true , address=185.167.204.205:4017 , piece_id=CV6LFILYHABDCW25TJJ7636AOKZ36JVLJJYXBNR7V4TZW4SJ7MUA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: read tcp 159.223.143.39:58588->185.167.204.205:4017: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034770401 (238h10m8.838129567s) [seg_pos={0 0}, upload=247902]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 18:26:08.139631321 (238h10m8.942990495s) [upload=247902]: Done uploading segments...
Feb 27 18:26:08.139695418 (238h10m8.943054606s) [upload=247902]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.126127427s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.047966ms remote:
"tcp:136.0.77.3:443" response headers:
Server=Storj Vary=Origin X-Amz-Request-Id=18982E2B110F99EB X-Request-Id=QiR1p4g5KyR X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:33:35 GMT Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/443023bb021edc61
FAIL (2026-02-27 18:33:35 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982E2B110F99EB</RequestId><HostId></HostId></Error>"
Duration: 9.084509ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.........................X......................X...........] duration:
2.51287346s trace:
https://jaeger.dev.storj.io/trace/7aa57c119569a500
FAIL (2026-02-27 18:33:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
KCFQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408102653 (238h9m9.211461838s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408567070 (238h9m9.211926247s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408707192 (238h9m9.212066378s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408968070 (238h9m9.212327244s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409367498 (238h9m9.212726677s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409377018 (238h9m9.212736185s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410004538 (238h9m9.213363725s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410656413 (238h9m9.214015608s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411331444 (238h9m9.214690672s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411885321 (238h9m9.215244506s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412446978 (238h9m9.21580616s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412960206 (238h9m9.216319389s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.833976270 (238h9m9.637335445s) [upload=247902]: Next returned nil segment
Feb 27 18:25:08.833993781 (238h9m9.637352963s) [upload=247902]: Waiting for error group managing segments...
Feb 27 18:25:09.002997616 (238h9m9.806356791s) [noise=true , address=81.171.14.70:4040 , piece_id=2MPY2M6YD4A35CALBOQBGARWAVYBWSALZU2P4JQNVTVU462KXOSQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.003988167 (238h9m9.807347334s) [noise=true , address=81.171.14.70:4037 , piece_id=O6CQWKOJPHOHZAO7PMTX74XFWNPWIDDDUE2S4NPO67A64TRGVLDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.007807120 (238h9m9.811166324s) [noise=true , address=81.171.14.70:4044 , piece_id=SX5RETWJXQ37MNWETTT3ISLFDEQMJ4LGWEWLA4SELKUG6L447YZQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.019462938 (238h9m9.822822115s) [noise=true , address=81.171.14.70:4039 , piece_id=VV7S6QSQSAMDYP2J4TWHDJNIGMTPEHG7N4737DETH4SSYKK3FKNA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.031013955 (238h9m9.834373141s) [noise=true , address=185.167.204.206:4050 , piece_id=NNXAXJUAW34ZSFMQZHSMONU2KSGJ4DNNNLGA3VCV7Q7JE4OXYUAA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.039442786 (238h9m9.84280196s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.050960211 (238h9m9.854319391s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.067324682 (238h9m9.870683857s) [noise=true , address=185.167.204.206:4067 , piece_id=LMTR6XVP77QRHIFJFPK2CCNTLFVKGCNFXBN2Z4AQQTE7YTJ2PDVA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.070489496 (238h9m9.87384866s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.071657801 (238h9m9.87501698s) [noise=true , address=185.167.204.206:4068 , piece_id=DRL3HHGWZQGLK5RWPTJFIOUERAFO3OH3FWGWCVCPLN3JAR7K7QPA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.072680314 (238h9m9.876039488s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.074685847 (238h9m9.878045033s) [noise=true , address=185.167.204.205:4008 , piece_id=CDKH6EYARWHKZIMNICX46Z3JSGBETOJEPGVGGESJ7LZZYGGBKKMQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.089811558 (238h9m9.893170735s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.098264180 (238h9m9.901623357s) [noise=true , address=185.167.204.206:4053 , piece_id=GOQ52CA2J4GSR7A5DXDDDMEDVHODA4FE5A6A2I3GMDSICFLSKCFQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101019345 (238h9m9.904378509s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101369094 (238h9m9.904728292s) [noise=true , address=185.167.204.205:4002 , piece_id=2DSKXMEVZMV5ECFAXVPHHSK3YS5LQI3LQNTGYD3DNCHKT74VGRSA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.102365507 (238h9m9.905724671s) [noise=true , address=185.167.204.206:4041 , piece_id=JYD734S7X4QLARQFX44G24S7YXDUX3GWJ4QOY5NHD2NY5XMLVVOA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.103504275 (238h9m9.90686345s) [noise=true , address=89.149.201.147:4008 , piece_id=HY4JE6TBQ3AAXVGQNXVKIMX6MNU45UMLPK2YEH4QXC4OBKPNEWNQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.106507437 (238h9m9.909866612s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.145089732 (238h9m9.948448907s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.473005969 (238h9m10.276365143s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:13.403656159 (238h9m14.207015334s) [noise=true , address=185.167.204.206:4059 , piece_id=RYP2J5M7ERW64IPIDS4YC6XSMXJJYFY2FIOHFFVJ4FMEZMFC45PQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:14.229596272 (238h9m15.032955449s) [noise=true , address=185.167.204.206:4071 , piece_id=S23HGNGGEWNCQP2TMJ3CH2HL4TQWMEYVDFHCYKFAHXBDAY4X6ILA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:17.197218248 (238h9m18.000577422s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:50.077929419 (238h9m50.881288595s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:26:08.034087443 (238h10m8.837446626s) [noise=true , address=89.149.201.147:4016 , piece_id=DRJK3ZPIZKZ44E6RPX6FGQZWPNMA7VMAIP6JXCB7IUQQ7QC5F7BQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034276836 (238h10m8.837636013s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 159.223.143.39:60508->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034741100 (238h10m8.83810028s) [noise=true , address=185.167.204.205:4017 , piece_id=CV6LFILYHABDCW25TJJ7636AOKZ36JVLJJYXBNR7V4TZW4SJ7MUA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: read tcp 159.223.143.39:58588->185.167.204.205:4017: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034770401 (238h10m8.838129567s) [seg_pos={0 0}, upload=247902]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 18:26:08.139631321 (238h10m8.942990495s) [upload=247902]: Done uploading segments...
Feb 27 18:26:08.139695418 (238h10m8.943054606s) [upload=247902]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.51290092s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.062372ms remote:
"tcp:136.0.77.3:443" response headers:
X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:33:20 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18982E276D15D92E X-Request-Id=gWgeCi1vwvb Accept-Ranges=bytes Content-Length=274 Server=Storj target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/2256a856da56f61e
FAIL (2026-02-27 18:33:20 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982E276D15D92E</RequestId><HostId></HostId></Error>"
Duration: 10.09298ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
27.705719ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Length=284 X-Amz-Request-Id=18982E25E93E0DF7 X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=ji4SLS3RWU3 Date=Fri, 27 Feb 2026 18:33:13 GMT target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/2480bce96d750af6
FAIL (2026-02-27 18:33:13 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982E25E93E0DF7</RequestId><HostId></HostId></Error>"
Duration: 27.730647ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
15.166493ms remote:
"tcp:136.0.77.3:443" response headers:
X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:33:02 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=18982E235AC7F430 X-Request-Id=iVAcPh92xmS Accept-Ranges=bytes target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/6ef839b2c0466d02
FAIL (2026-02-27 18:33:02 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982E235AC7F430</RequestId><HostId></HostId></Error>"
Duration: 15.268887ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.467044ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18982E1F4DEDBFE3 X-Request-Id=eqj86FwJX71 X-Xss-Protection=1; mode=block Server=Storj Date=Fri, 27 Feb 2026 18:32:45 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/53d2c93630781bc7
FAIL (2026-02-27 18:32:45 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982E1F4DEDBFE3</RequestId><HostId></HostId></Error>"
Duration: 11.558784ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..........................X......................X..........] duration:
2.544089211s trace:
https://jaeger.dev.storj.io/trace/7aa57c119569a500
FAIL (2026-02-27 18:32:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
KCFQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408102653 (238h9m9.211461838s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408567070 (238h9m9.211926247s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408707192 (238h9m9.212066378s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408968070 (238h9m9.212327244s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409367498 (238h9m9.212726677s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409377018 (238h9m9.212736185s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410004538 (238h9m9.213363725s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410656413 (238h9m9.214015608s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411331444 (238h9m9.214690672s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411885321 (238h9m9.215244506s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412446978 (238h9m9.21580616s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412960206 (238h9m9.216319389s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.833976270 (238h9m9.637335445s) [upload=247902]: Next returned nil segment
Feb 27 18:25:08.833993781 (238h9m9.637352963s) [upload=247902]: Waiting for error group managing segments...
Feb 27 18:25:09.002997616 (238h9m9.806356791s) [noise=true , address=81.171.14.70:4040 , piece_id=2MPY2M6YD4A35CALBOQBGARWAVYBWSALZU2P4JQNVTVU462KXOSQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.003988167 (238h9m9.807347334s) [noise=true , address=81.171.14.70:4037 , piece_id=O6CQWKOJPHOHZAO7PMTX74XFWNPWIDDDUE2S4NPO67A64TRGVLDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.007807120 (238h9m9.811166324s) [noise=true , address=81.171.14.70:4044 , piece_id=SX5RETWJXQ37MNWETTT3ISLFDEQMJ4LGWEWLA4SELKUG6L447YZQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.019462938 (238h9m9.822822115s) [noise=true , address=81.171.14.70:4039 , piece_id=VV7S6QSQSAMDYP2J4TWHDJNIGMTPEHG7N4737DETH4SSYKK3FKNA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.031013955 (238h9m9.834373141s) [noise=true , address=185.167.204.206:4050 , piece_id=NNXAXJUAW34ZSFMQZHSMONU2KSGJ4DNNNLGA3VCV7Q7JE4OXYUAA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.039442786 (238h9m9.84280196s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.050960211 (238h9m9.854319391s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.067324682 (238h9m9.870683857s) [noise=true , address=185.167.204.206:4067 , piece_id=LMTR6XVP77QRHIFJFPK2CCNTLFVKGCNFXBN2Z4AQQTE7YTJ2PDVA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.070489496 (238h9m9.87384866s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.071657801 (238h9m9.87501698s) [noise=true , address=185.167.204.206:4068 , piece_id=DRL3HHGWZQGLK5RWPTJFIOUERAFO3OH3FWGWCVCPLN3JAR7K7QPA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.072680314 (238h9m9.876039488s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.074685847 (238h9m9.878045033s) [noise=true , address=185.167.204.205:4008 , piece_id=CDKH6EYARWHKZIMNICX46Z3JSGBETOJEPGVGGESJ7LZZYGGBKKMQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.089811558 (238h9m9.893170735s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.098264180 (238h9m9.901623357s) [noise=true , address=185.167.204.206:4053 , piece_id=GOQ52CA2J4GSR7A5DXDDDMEDVHODA4FE5A6A2I3GMDSICFLSKCFQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101019345 (238h9m9.904378509s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101369094 (238h9m9.904728292s) [noise=true , address=185.167.204.205:4002 , piece_id=2DSKXMEVZMV5ECFAXVPHHSK3YS5LQI3LQNTGYD3DNCHKT74VGRSA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.102365507 (238h9m9.905724671s) [noise=true , address=185.167.204.206:4041 , piece_id=JYD734S7X4QLARQFX44G24S7YXDUX3GWJ4QOY5NHD2NY5XMLVVOA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.103504275 (238h9m9.90686345s) [noise=true , address=89.149.201.147:4008 , piece_id=HY4JE6TBQ3AAXVGQNXVKIMX6MNU45UMLPK2YEH4QXC4OBKPNEWNQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.106507437 (238h9m9.909866612s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.145089732 (238h9m9.948448907s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.473005969 (238h9m10.276365143s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:13.403656159 (238h9m14.207015334s) [noise=true , address=185.167.204.206:4059 , piece_id=RYP2J5M7ERW64IPIDS4YC6XSMXJJYFY2FIOHFFVJ4FMEZMFC45PQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:14.229596272 (238h9m15.032955449s) [noise=true , address=185.167.204.206:4071 , piece_id=S23HGNGGEWNCQP2TMJ3CH2HL4TQWMEYVDFHCYKFAHXBDAY4X6ILA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:17.197218248 (238h9m18.000577422s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:50.077929419 (238h9m50.881288595s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:26:08.034087443 (238h10m8.837446626s) [noise=true , address=89.149.201.147:4016 , piece_id=DRJK3ZPIZKZ44E6RPX6FGQZWPNMA7VMAIP6JXCB7IUQQ7QC5F7BQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034276836 (238h10m8.837636013s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 159.223.143.39:60508->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034741100 (238h10m8.83810028s) [noise=true , address=185.167.204.205:4017 , piece_id=CV6LFILYHABDCW25TJJ7636AOKZ36JVLJJYXBNR7V4TZW4SJ7MUA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: read tcp 159.223.143.39:58588->185.167.204.205:4017: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034770401 (238h10m8.838129567s) [seg_pos={0 0}, upload=247902]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 18:26:08.139631321 (238h10m8.942990495s) [upload=247902]: Done uploading segments...
Feb 27 18:26:08.139695418 (238h10m8.943054606s) [upload=247902]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.544123022s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.630985ms remote:
"tcp:136.0.77.3:443" response headers:
X-Amz-Request-Id=18982E1CFD7FC768 X-Request-Id=c4dGTqmZJny Date=Fri, 27 Feb 2026 18:32:35 GMT Content-Type=application/xml Vary=Origin X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Server=Storj target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/6bce36dbaa61a6ad
FAIL (2026-02-27 18:32:35 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982E1CFD7FC768</RequestId><HostId></HostId></Error>"
Duration: 9.776133ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
28.661521ms remote:
"tcp:45.38.24.2:443" response headers:
Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=18982E1A92AA4158 X-Request-Id=UfqbpNqJJTZ Date=Fri, 27 Feb 2026 18:32:25 GMT Content-Length=284 X-Xss-Protection=1; mode=block target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/7fa95fa1cd03b750
FAIL (2026-02-27 18:32:25 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982E1A92AA4158</RequestId><HostId></HostId></Error>"
Duration: 28.678296ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.779752ms remote:
"tcp:136.0.77.3:443" response headers:
Vary=Origin X-Amz-Request-Id=18982E1914170E35 X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:32:18 GMT Content-Length=274 Server=Storj X-Request-Id=68zPusPZvCr Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/3cb376434b4546c0
FAIL (2026-02-27 18:32:18 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982E1914170E35</RequestId><HostId></HostId></Error>"
Duration: 9.810033ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
13.68907ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Amz-Request-Id=18982E18B3D59110 X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:32:17 GMT Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml X-Request-Id=B7DxXXYhMVR target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/47d175c2195fae83
FAIL (2026-02-27 18:32:17 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982E18B3D59110</RequestId><HostId></HostId></Error>"
Duration: 13.71902ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...........................X......................X.........] duration:
2.10539931s trace:
https://jaeger.dev.storj.io/trace/7aa57c119569a500
FAIL (2026-02-27 18:32:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
KCFQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408102653 (238h9m9.211461838s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408567070 (238h9m9.211926247s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408707192 (238h9m9.212066378s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408968070 (238h9m9.212327244s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409367498 (238h9m9.212726677s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409377018 (238h9m9.212736185s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410004538 (238h9m9.213363725s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410656413 (238h9m9.214015608s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411331444 (238h9m9.214690672s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411885321 (238h9m9.215244506s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412446978 (238h9m9.21580616s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412960206 (238h9m9.216319389s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.833976270 (238h9m9.637335445s) [upload=247902]: Next returned nil segment
Feb 27 18:25:08.833993781 (238h9m9.637352963s) [upload=247902]: Waiting for error group managing segments...
Feb 27 18:25:09.002997616 (238h9m9.806356791s) [noise=true , address=81.171.14.70:4040 , piece_id=2MPY2M6YD4A35CALBOQBGARWAVYBWSALZU2P4JQNVTVU462KXOSQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.003988167 (238h9m9.807347334s) [noise=true , address=81.171.14.70:4037 , piece_id=O6CQWKOJPHOHZAO7PMTX74XFWNPWIDDDUE2S4NPO67A64TRGVLDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.007807120 (238h9m9.811166324s) [noise=true , address=81.171.14.70:4044 , piece_id=SX5RETWJXQ37MNWETTT3ISLFDEQMJ4LGWEWLA4SELKUG6L447YZQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.019462938 (238h9m9.822822115s) [noise=true , address=81.171.14.70:4039 , piece_id=VV7S6QSQSAMDYP2J4TWHDJNIGMTPEHG7N4737DETH4SSYKK3FKNA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.031013955 (238h9m9.834373141s) [noise=true , address=185.167.204.206:4050 , piece_id=NNXAXJUAW34ZSFMQZHSMONU2KSGJ4DNNNLGA3VCV7Q7JE4OXYUAA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.039442786 (238h9m9.84280196s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.050960211 (238h9m9.854319391s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.067324682 (238h9m9.870683857s) [noise=true , address=185.167.204.206:4067 , piece_id=LMTR6XVP77QRHIFJFPK2CCNTLFVKGCNFXBN2Z4AQQTE7YTJ2PDVA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.070489496 (238h9m9.87384866s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.071657801 (238h9m9.87501698s) [noise=true , address=185.167.204.206:4068 , piece_id=DRL3HHGWZQGLK5RWPTJFIOUERAFO3OH3FWGWCVCPLN3JAR7K7QPA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.072680314 (238h9m9.876039488s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.074685847 (238h9m9.878045033s) [noise=true , address=185.167.204.205:4008 , piece_id=CDKH6EYARWHKZIMNICX46Z3JSGBETOJEPGVGGESJ7LZZYGGBKKMQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.089811558 (238h9m9.893170735s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.098264180 (238h9m9.901623357s) [noise=true , address=185.167.204.206:4053 , piece_id=GOQ52CA2J4GSR7A5DXDDDMEDVHODA4FE5A6A2I3GMDSICFLSKCFQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101019345 (238h9m9.904378509s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101369094 (238h9m9.904728292s) [noise=true , address=185.167.204.205:4002 , piece_id=2DSKXMEVZMV5ECFAXVPHHSK3YS5LQI3LQNTGYD3DNCHKT74VGRSA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.102365507 (238h9m9.905724671s) [noise=true , address=185.167.204.206:4041 , piece_id=JYD734S7X4QLARQFX44G24S7YXDUX3GWJ4QOY5NHD2NY5XMLVVOA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.103504275 (238h9m9.90686345s) [noise=true , address=89.149.201.147:4008 , piece_id=HY4JE6TBQ3AAXVGQNXVKIMX6MNU45UMLPK2YEH4QXC4OBKPNEWNQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.106507437 (238h9m9.909866612s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.145089732 (238h9m9.948448907s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.473005969 (238h9m10.276365143s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:13.403656159 (238h9m14.207015334s) [noise=true , address=185.167.204.206:4059 , piece_id=RYP2J5M7ERW64IPIDS4YC6XSMXJJYFY2FIOHFFVJ4FMEZMFC45PQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:14.229596272 (238h9m15.032955449s) [noise=true , address=185.167.204.206:4071 , piece_id=S23HGNGGEWNCQP2TMJ3CH2HL4TQWMEYVDFHCYKFAHXBDAY4X6ILA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:17.197218248 (238h9m18.000577422s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:50.077929419 (238h9m50.881288595s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:26:08.034087443 (238h10m8.837446626s) [noise=true , address=89.149.201.147:4016 , piece_id=DRJK3ZPIZKZ44E6RPX6FGQZWPNMA7VMAIP6JXCB7IUQQ7QC5F7BQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034276836 (238h10m8.837636013s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 159.223.143.39:60508->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034741100 (238h10m8.83810028s) [noise=true , address=185.167.204.205:4017 , piece_id=CV6LFILYHABDCW25TJJ7636AOKZ36JVLJJYXBNR7V4TZW4SJ7MUA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: read tcp 159.223.143.39:58588->185.167.204.205:4017: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034770401 (238h10m8.838129567s) [seg_pos={0 0}, upload=247902]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 18:26:08.139631321 (238h10m8.942990495s) [upload=247902]: Done uploading segments...
Feb 27 18:26:08.139695418 (238h10m8.943054606s) [upload=247902]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.105428717s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
27.804184ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Length=284 Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=18982E106E0AEBE5 X-Request-Id=gtXZG1Zp5KA X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Date=Fri, 27 Feb 2026 18:31:41 GMT target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/2f932741ea80ce14
FAIL (2026-02-27 18:31:41 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982E106E0AEBE5</RequestId><HostId></HostId></Error>"
Duration: 27.850005ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
24.657079ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Request-Id=SSHpSiCWBqg X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:31:37 GMT Accept-Ranges=bytes Server=Storj X-Amz-Request-Id=18982E0F698B2376 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/1e3fa54e3a102cb9
FAIL (2026-02-27 18:31:37 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982E0F698B2376</RequestId><HostId></HostId></Error>"
Duration: 24.672583ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.299275ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Security-Policy=block-all-mixed-content X-Request-Id=RCbgdA2cBba X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:31:14 GMT Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=18982E0A0F07BE8C target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/2a8463ac838ff65e
FAIL (2026-02-27 18:31:14 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982E0A0F07BE8C</RequestId><HostId></HostId></Error>"
Duration: 10.322881ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [............................X......................X........] duration:
2.123807323s trace:
https://jaeger.dev.storj.io/trace/7aa57c119569a500
FAIL (2026-02-27 18:31:11 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
KCFQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408102653 (238h9m9.211461838s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408567070 (238h9m9.211926247s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408707192 (238h9m9.212066378s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408968070 (238h9m9.212327244s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409367498 (238h9m9.212726677s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409377018 (238h9m9.212736185s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410004538 (238h9m9.213363725s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410656413 (238h9m9.214015608s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411331444 (238h9m9.214690672s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411885321 (238h9m9.215244506s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412446978 (238h9m9.21580616s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412960206 (238h9m9.216319389s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.833976270 (238h9m9.637335445s) [upload=247902]: Next returned nil segment
Feb 27 18:25:08.833993781 (238h9m9.637352963s) [upload=247902]: Waiting for error group managing segments...
Feb 27 18:25:09.002997616 (238h9m9.806356791s) [noise=true , address=81.171.14.70:4040 , piece_id=2MPY2M6YD4A35CALBOQBGARWAVYBWSALZU2P4JQNVTVU462KXOSQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.003988167 (238h9m9.807347334s) [noise=true , address=81.171.14.70:4037 , piece_id=O6CQWKOJPHOHZAO7PMTX74XFWNPWIDDDUE2S4NPO67A64TRGVLDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.007807120 (238h9m9.811166324s) [noise=true , address=81.171.14.70:4044 , piece_id=SX5RETWJXQ37MNWETTT3ISLFDEQMJ4LGWEWLA4SELKUG6L447YZQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.019462938 (238h9m9.822822115s) [noise=true , address=81.171.14.70:4039 , piece_id=VV7S6QSQSAMDYP2J4TWHDJNIGMTPEHG7N4737DETH4SSYKK3FKNA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.031013955 (238h9m9.834373141s) [noise=true , address=185.167.204.206:4050 , piece_id=NNXAXJUAW34ZSFMQZHSMONU2KSGJ4DNNNLGA3VCV7Q7JE4OXYUAA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.039442786 (238h9m9.84280196s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.050960211 (238h9m9.854319391s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.067324682 (238h9m9.870683857s) [noise=true , address=185.167.204.206:4067 , piece_id=LMTR6XVP77QRHIFJFPK2CCNTLFVKGCNFXBN2Z4AQQTE7YTJ2PDVA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.070489496 (238h9m9.87384866s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.071657801 (238h9m9.87501698s) [noise=true , address=185.167.204.206:4068 , piece_id=DRL3HHGWZQGLK5RWPTJFIOUERAFO3OH3FWGWCVCPLN3JAR7K7QPA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.072680314 (238h9m9.876039488s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.074685847 (238h9m9.878045033s) [noise=true , address=185.167.204.205:4008 , piece_id=CDKH6EYARWHKZIMNICX46Z3JSGBETOJEPGVGGESJ7LZZYGGBKKMQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.089811558 (238h9m9.893170735s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.098264180 (238h9m9.901623357s) [noise=true , address=185.167.204.206:4053 , piece_id=GOQ52CA2J4GSR7A5DXDDDMEDVHODA4FE5A6A2I3GMDSICFLSKCFQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101019345 (238h9m9.904378509s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101369094 (238h9m9.904728292s) [noise=true , address=185.167.204.205:4002 , piece_id=2DSKXMEVZMV5ECFAXVPHHSK3YS5LQI3LQNTGYD3DNCHKT74VGRSA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.102365507 (238h9m9.905724671s) [noise=true , address=185.167.204.206:4041 , piece_id=JYD734S7X4QLARQFX44G24S7YXDUX3GWJ4QOY5NHD2NY5XMLVVOA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.103504275 (238h9m9.90686345s) [noise=true , address=89.149.201.147:4008 , piece_id=HY4JE6TBQ3AAXVGQNXVKIMX6MNU45UMLPK2YEH4QXC4OBKPNEWNQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.106507437 (238h9m9.909866612s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.145089732 (238h9m9.948448907s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.473005969 (238h9m10.276365143s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:13.403656159 (238h9m14.207015334s) [noise=true , address=185.167.204.206:4059 , piece_id=RYP2J5M7ERW64IPIDS4YC6XSMXJJYFY2FIOHFFVJ4FMEZMFC45PQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:14.229596272 (238h9m15.032955449s) [noise=true , address=185.167.204.206:4071 , piece_id=S23HGNGGEWNCQP2TMJ3CH2HL4TQWMEYVDFHCYKFAHXBDAY4X6ILA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:17.197218248 (238h9m18.000577422s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:50.077929419 (238h9m50.881288595s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:26:08.034087443 (238h10m8.837446626s) [noise=true , address=89.149.201.147:4016 , piece_id=DRJK3ZPIZKZ44E6RPX6FGQZWPNMA7VMAIP6JXCB7IUQQ7QC5F7BQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034276836 (238h10m8.837636013s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 159.223.143.39:60508->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034741100 (238h10m8.83810028s) [noise=true , address=185.167.204.205:4017 , piece_id=CV6LFILYHABDCW25TJJ7636AOKZ36JVLJJYXBNR7V4TZW4SJ7MUA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: read tcp 159.223.143.39:58588->185.167.204.205:4017: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034770401 (238h10m8.838129567s) [seg_pos={0 0}, upload=247902]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 18:26:08.139631321 (238h10m8.942990495s) [upload=247902]: Done uploading segments...
Feb 27 18:26:08.139695418 (238h10m8.943054606s) [upload=247902]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.123837777s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.576725ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Amz-Request-Id=18982E09635ADFA0 X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:31:11 GMT Content-Length=274 Content-Type=application/xml X-Request-Id=2A5CXrG2Bbe target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/5347b72821f0c829
FAIL (2026-02-27 18:31:11 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982E09635ADFA0</RequestId><HostId></HostId></Error>"
Duration: 9.600386ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
31.718774ms remote:
"tcp:45.38.24.2:443" response headers:
Server=Storj Vary=Origin X-Request-Id=DWtkfbiDY4Y Date=Fri, 27 Feb 2026 18:31:02 GMT Accept-Ranges=bytes Content-Length=284 X-Amz-Request-Id=18982E076E2E3AFB X-Xss-Protection=1; mode=block Content-Security-Policy=block-all-mixed-content Content-Type=application/xml target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/4d1234b33f9a8e90
FAIL (2026-02-27 18:31:02 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982E076E2E3AFB</RequestId><HostId></HostId></Error>"
Duration: 31.842357ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
18.978851ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=18982E0637419715 X-Request-Id=gzVdM8oAhLt X-Xss-Protection=1; mode=block Content-Security-Policy=block-all-mixed-content Date=Fri, 27 Feb 2026 18:30:57 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/4096b4b00fa61ae0
FAIL (2026-02-27 18:30:57 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982E0637419715</RequestId><HostId></HostId></Error>"
Duration: 18.999857ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.............................X......................X.......] duration:
2.197448761s trace:
https://jaeger.dev.storj.io/trace/7aa57c119569a500
FAIL (2026-02-27 18:30:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
KCFQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408102653 (238h9m9.211461838s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408567070 (238h9m9.211926247s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408707192 (238h9m9.212066378s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408968070 (238h9m9.212327244s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409367498 (238h9m9.212726677s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409377018 (238h9m9.212736185s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410004538 (238h9m9.213363725s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410656413 (238h9m9.214015608s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411331444 (238h9m9.214690672s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411885321 (238h9m9.215244506s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412446978 (238h9m9.21580616s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412960206 (238h9m9.216319389s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.833976270 (238h9m9.637335445s) [upload=247902]: Next returned nil segment
Feb 27 18:25:08.833993781 (238h9m9.637352963s) [upload=247902]: Waiting for error group managing segments...
Feb 27 18:25:09.002997616 (238h9m9.806356791s) [noise=true , address=81.171.14.70:4040 , piece_id=2MPY2M6YD4A35CALBOQBGARWAVYBWSALZU2P4JQNVTVU462KXOSQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.003988167 (238h9m9.807347334s) [noise=true , address=81.171.14.70:4037 , piece_id=O6CQWKOJPHOHZAO7PMTX74XFWNPWIDDDUE2S4NPO67A64TRGVLDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.007807120 (238h9m9.811166324s) [noise=true , address=81.171.14.70:4044 , piece_id=SX5RETWJXQ37MNWETTT3ISLFDEQMJ4LGWEWLA4SELKUG6L447YZQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.019462938 (238h9m9.822822115s) [noise=true , address=81.171.14.70:4039 , piece_id=VV7S6QSQSAMDYP2J4TWHDJNIGMTPEHG7N4737DETH4SSYKK3FKNA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.031013955 (238h9m9.834373141s) [noise=true , address=185.167.204.206:4050 , piece_id=NNXAXJUAW34ZSFMQZHSMONU2KSGJ4DNNNLGA3VCV7Q7JE4OXYUAA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.039442786 (238h9m9.84280196s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.050960211 (238h9m9.854319391s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.067324682 (238h9m9.870683857s) [noise=true , address=185.167.204.206:4067 , piece_id=LMTR6XVP77QRHIFJFPK2CCNTLFVKGCNFXBN2Z4AQQTE7YTJ2PDVA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.070489496 (238h9m9.87384866s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.071657801 (238h9m9.87501698s) [noise=true , address=185.167.204.206:4068 , piece_id=DRL3HHGWZQGLK5RWPTJFIOUERAFO3OH3FWGWCVCPLN3JAR7K7QPA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.072680314 (238h9m9.876039488s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.074685847 (238h9m9.878045033s) [noise=true , address=185.167.204.205:4008 , piece_id=CDKH6EYARWHKZIMNICX46Z3JSGBETOJEPGVGGESJ7LZZYGGBKKMQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.089811558 (238h9m9.893170735s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.098264180 (238h9m9.901623357s) [noise=true , address=185.167.204.206:4053 , piece_id=GOQ52CA2J4GSR7A5DXDDDMEDVHODA4FE5A6A2I3GMDSICFLSKCFQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101019345 (238h9m9.904378509s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101369094 (238h9m9.904728292s) [noise=true , address=185.167.204.205:4002 , piece_id=2DSKXMEVZMV5ECFAXVPHHSK3YS5LQI3LQNTGYD3DNCHKT74VGRSA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.102365507 (238h9m9.905724671s) [noise=true , address=185.167.204.206:4041 , piece_id=JYD734S7X4QLARQFX44G24S7YXDUX3GWJ4QOY5NHD2NY5XMLVVOA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.103504275 (238h9m9.90686345s) [noise=true , address=89.149.201.147:4008 , piece_id=HY4JE6TBQ3AAXVGQNXVKIMX6MNU45UMLPK2YEH4QXC4OBKPNEWNQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.106507437 (238h9m9.909866612s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.145089732 (238h9m9.948448907s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.473005969 (238h9m10.276365143s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:13.403656159 (238h9m14.207015334s) [noise=true , address=185.167.204.206:4059 , piece_id=RYP2J5M7ERW64IPIDS4YC6XSMXJJYFY2FIOHFFVJ4FMEZMFC45PQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:14.229596272 (238h9m15.032955449s) [noise=true , address=185.167.204.206:4071 , piece_id=S23HGNGGEWNCQP2TMJ3CH2HL4TQWMEYVDFHCYKFAHXBDAY4X6ILA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:17.197218248 (238h9m18.000577422s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:50.077929419 (238h9m50.881288595s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:26:08.034087443 (238h10m8.837446626s) [noise=true , address=89.149.201.147:4016 , piece_id=DRJK3ZPIZKZ44E6RPX6FGQZWPNMA7VMAIP6JXCB7IUQQ7QC5F7BQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034276836 (238h10m8.837636013s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 159.223.143.39:60508->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034741100 (238h10m8.83810028s) [noise=true , address=185.167.204.205:4017 , piece_id=CV6LFILYHABDCW25TJJ7636AOKZ36JVLJJYXBNR7V4TZW4SJ7MUA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: read tcp 159.223.143.39:58588->185.167.204.205:4017: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034770401 (238h10m8.838129567s) [seg_pos={0 0}, upload=247902]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 18:26:08.139631321 (238h10m8.942990495s) [upload=247902]: Done uploading segments...
Feb 27 18:26:08.139695418 (238h10m8.943054606s) [upload=247902]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.197483195s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
31.738057ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=18982E0027BBE0B6 X-Request-Id=3SuUBeyBxAF Accept-Ranges=bytes Content-Length=284 Content-Security-Policy=block-all-mixed-content X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:30:31 GMT target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/78694e371ead7b20
FAIL (2026-02-27 18:30:31 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982E0027BBE0B6</RequestId><HostId></HostId></Error>"
Duration: 31.78685ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
14.857493ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Length=274 Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18982DFBDFA4F832 Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=f4rJcA8HWpX X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:30:13 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/314fb88ab0b58ed5
FAIL (2026-02-27 18:30:13 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DFBDFA4F832</RequestId><HostId></HostId></Error>"
Duration: 14.893017ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
15.03435ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Request-Id=BFiJetv5jhE X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:30:10 GMT Content-Length=274 Server=Storj X-Amz-Request-Id=18982DFB47086969 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/381cf652ed85ce61
FAIL (2026-02-27 18:30:10 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DFB47086969</RequestId><HostId></HostId></Error>"
Duration: 15.076391ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
15.308069ms remote:
"tcp:136.0.77.3:443" response headers:
Vary=Origin X-Amz-Request-Id=18982DF7B90F3A3A X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:29:55 GMT Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content X-Request-Id=8MGRLCWJJCt Content-Type=application/xml Server=Storj target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/6e409c4b8a2073c0
FAIL (2026-02-27 18:29:55 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DF7B90F3A3A</RequestId><HostId></HostId></Error>"
Duration: 15.403439ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..............................X......................X......] duration:
2.041008587s trace:
https://jaeger.dev.storj.io/trace/7aa57c119569a500
FAIL (2026-02-27 18:29:35 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
KCFQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408102653 (238h9m9.211461838s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408567070 (238h9m9.211926247s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408707192 (238h9m9.212066378s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408968070 (238h9m9.212327244s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409367498 (238h9m9.212726677s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409377018 (238h9m9.212736185s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410004538 (238h9m9.213363725s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410656413 (238h9m9.214015608s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411331444 (238h9m9.214690672s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411885321 (238h9m9.215244506s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412446978 (238h9m9.21580616s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412960206 (238h9m9.216319389s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.833976270 (238h9m9.637335445s) [upload=247902]: Next returned nil segment
Feb 27 18:25:08.833993781 (238h9m9.637352963s) [upload=247902]: Waiting for error group managing segments...
Feb 27 18:25:09.002997616 (238h9m9.806356791s) [noise=true , address=81.171.14.70:4040 , piece_id=2MPY2M6YD4A35CALBOQBGARWAVYBWSALZU2P4JQNVTVU462KXOSQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.003988167 (238h9m9.807347334s) [noise=true , address=81.171.14.70:4037 , piece_id=O6CQWKOJPHOHZAO7PMTX74XFWNPWIDDDUE2S4NPO67A64TRGVLDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.007807120 (238h9m9.811166324s) [noise=true , address=81.171.14.70:4044 , piece_id=SX5RETWJXQ37MNWETTT3ISLFDEQMJ4LGWEWLA4SELKUG6L447YZQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.019462938 (238h9m9.822822115s) [noise=true , address=81.171.14.70:4039 , piece_id=VV7S6QSQSAMDYP2J4TWHDJNIGMTPEHG7N4737DETH4SSYKK3FKNA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.031013955 (238h9m9.834373141s) [noise=true , address=185.167.204.206:4050 , piece_id=NNXAXJUAW34ZSFMQZHSMONU2KSGJ4DNNNLGA3VCV7Q7JE4OXYUAA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.039442786 (238h9m9.84280196s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.050960211 (238h9m9.854319391s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.067324682 (238h9m9.870683857s) [noise=true , address=185.167.204.206:4067 , piece_id=LMTR6XVP77QRHIFJFPK2CCNTLFVKGCNFXBN2Z4AQQTE7YTJ2PDVA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.070489496 (238h9m9.87384866s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.071657801 (238h9m9.87501698s) [noise=true , address=185.167.204.206:4068 , piece_id=DRL3HHGWZQGLK5RWPTJFIOUERAFO3OH3FWGWCVCPLN3JAR7K7QPA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.072680314 (238h9m9.876039488s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.074685847 (238h9m9.878045033s) [noise=true , address=185.167.204.205:4008 , piece_id=CDKH6EYARWHKZIMNICX46Z3JSGBETOJEPGVGGESJ7LZZYGGBKKMQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.089811558 (238h9m9.893170735s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.098264180 (238h9m9.901623357s) [noise=true , address=185.167.204.206:4053 , piece_id=GOQ52CA2J4GSR7A5DXDDDMEDVHODA4FE5A6A2I3GMDSICFLSKCFQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101019345 (238h9m9.904378509s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101369094 (238h9m9.904728292s) [noise=true , address=185.167.204.205:4002 , piece_id=2DSKXMEVZMV5ECFAXVPHHSK3YS5LQI3LQNTGYD3DNCHKT74VGRSA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.102365507 (238h9m9.905724671s) [noise=true , address=185.167.204.206:4041 , piece_id=JYD734S7X4QLARQFX44G24S7YXDUX3GWJ4QOY5NHD2NY5XMLVVOA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.103504275 (238h9m9.90686345s) [noise=true , address=89.149.201.147:4008 , piece_id=HY4JE6TBQ3AAXVGQNXVKIMX6MNU45UMLPK2YEH4QXC4OBKPNEWNQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.106507437 (238h9m9.909866612s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.145089732 (238h9m9.948448907s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.473005969 (238h9m10.276365143s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:13.403656159 (238h9m14.207015334s) [noise=true , address=185.167.204.206:4059 , piece_id=RYP2J5M7ERW64IPIDS4YC6XSMXJJYFY2FIOHFFVJ4FMEZMFC45PQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:14.229596272 (238h9m15.032955449s) [noise=true , address=185.167.204.206:4071 , piece_id=S23HGNGGEWNCQP2TMJ3CH2HL4TQWMEYVDFHCYKFAHXBDAY4X6ILA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:17.197218248 (238h9m18.000577422s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:50.077929419 (238h9m50.881288595s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:26:08.034087443 (238h10m8.837446626s) [noise=true , address=89.149.201.147:4016 , piece_id=DRJK3ZPIZKZ44E6RPX6FGQZWPNMA7VMAIP6JXCB7IUQQ7QC5F7BQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034276836 (238h10m8.837636013s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 159.223.143.39:60508->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034741100 (238h10m8.83810028s) [noise=true , address=185.167.204.205:4017 , piece_id=CV6LFILYHABDCW25TJJ7636AOKZ36JVLJJYXBNR7V4TZW4SJ7MUA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: read tcp 159.223.143.39:58588->185.167.204.205:4017: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034770401 (238h10m8.838129567s) [seg_pos={0 0}, upload=247902]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 18:26:08.139631321 (238h10m8.942990495s) [upload=247902]: Done uploading segments...
Feb 27 18:26:08.139695418 (238h10m8.943054606s) [upload=247902]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.041033664s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
14.417015ms remote:
"tcp:136.0.77.3:443" response headers:
Vary=Origin X-Amz-Request-Id=18982DF296277EDF Date=Fri, 27 Feb 2026 18:29:33 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Request-Id=NtMW6RWBYy6 X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=274 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/29d85b6b9e78a1c1
FAIL (2026-02-27 18:29:33 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DF296277EDF</RequestId><HostId></HostId></Error>"
Duration: 14.547334ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.532448ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Length=284 Content-Type=application/xml Server=Storj X-Amz-Request-Id=18982DF25C93ACA6 X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:29:32 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Vary=Origin X-Request-Id=7wwUHfvoFgR target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/3141b06ad28a3c43
FAIL (2026-02-27 18:29:32 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DF25C93ACA6</RequestId><HostId></HostId></Error>"
Duration: 30.555993ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.956979ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Security-Policy=block-all-mixed-content Vary=Origin X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:29:14 GMT Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml Server=Storj X-Amz-Request-Id=18982DEE4E109284 X-Request-Id=dC7A5hhJij4 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/6feda6efece2878c
FAIL (2026-02-27 18:29:14 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DEE4E109284</RequestId><HostId></HostId></Error>"
Duration: 10.000638ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
12.487343ms remote:
"tcp:136.0.77.3:443" response headers:
X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:28:56 GMT Content-Length=274 Content-Type=application/xml Vary=Origin Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Server=Storj X-Amz-Request-Id=18982DE9F051636C X-Request-Id=22Sy8P5Uvzq target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/7ab841356736576
FAIL (2026-02-27 18:28:56 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DE9F051636C</RequestId><HostId></HostId></Error>"
Duration: 12.516251ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...............................X......................X.....] duration:
2.138427491s trace:
https://jaeger.dev.storj.io/trace/7aa57c119569a500
FAIL (2026-02-27 18:28:53 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
KCFQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408102653 (238h9m9.211461838s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408567070 (238h9m9.211926247s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408707192 (238h9m9.212066378s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408968070 (238h9m9.212327244s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409367498 (238h9m9.212726677s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409377018 (238h9m9.212736185s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410004538 (238h9m9.213363725s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410656413 (238h9m9.214015608s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411331444 (238h9m9.214690672s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411885321 (238h9m9.215244506s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412446978 (238h9m9.21580616s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412960206 (238h9m9.216319389s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.833976270 (238h9m9.637335445s) [upload=247902]: Next returned nil segment
Feb 27 18:25:08.833993781 (238h9m9.637352963s) [upload=247902]: Waiting for error group managing segments...
Feb 27 18:25:09.002997616 (238h9m9.806356791s) [noise=true , address=81.171.14.70:4040 , piece_id=2MPY2M6YD4A35CALBOQBGARWAVYBWSALZU2P4JQNVTVU462KXOSQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.003988167 (238h9m9.807347334s) [noise=true , address=81.171.14.70:4037 , piece_id=O6CQWKOJPHOHZAO7PMTX74XFWNPWIDDDUE2S4NPO67A64TRGVLDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.007807120 (238h9m9.811166324s) [noise=true , address=81.171.14.70:4044 , piece_id=SX5RETWJXQ37MNWETTT3ISLFDEQMJ4LGWEWLA4SELKUG6L447YZQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.019462938 (238h9m9.822822115s) [noise=true , address=81.171.14.70:4039 , piece_id=VV7S6QSQSAMDYP2J4TWHDJNIGMTPEHG7N4737DETH4SSYKK3FKNA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.031013955 (238h9m9.834373141s) [noise=true , address=185.167.204.206:4050 , piece_id=NNXAXJUAW34ZSFMQZHSMONU2KSGJ4DNNNLGA3VCV7Q7JE4OXYUAA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.039442786 (238h9m9.84280196s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.050960211 (238h9m9.854319391s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.067324682 (238h9m9.870683857s) [noise=true , address=185.167.204.206:4067 , piece_id=LMTR6XVP77QRHIFJFPK2CCNTLFVKGCNFXBN2Z4AQQTE7YTJ2PDVA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.070489496 (238h9m9.87384866s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.071657801 (238h9m9.87501698s) [noise=true , address=185.167.204.206:4068 , piece_id=DRL3HHGWZQGLK5RWPTJFIOUERAFO3OH3FWGWCVCPLN3JAR7K7QPA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.072680314 (238h9m9.876039488s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.074685847 (238h9m9.878045033s) [noise=true , address=185.167.204.205:4008 , piece_id=CDKH6EYARWHKZIMNICX46Z3JSGBETOJEPGVGGESJ7LZZYGGBKKMQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.089811558 (238h9m9.893170735s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.098264180 (238h9m9.901623357s) [noise=true , address=185.167.204.206:4053 , piece_id=GOQ52CA2J4GSR7A5DXDDDMEDVHODA4FE5A6A2I3GMDSICFLSKCFQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101019345 (238h9m9.904378509s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101369094 (238h9m9.904728292s) [noise=true , address=185.167.204.205:4002 , piece_id=2DSKXMEVZMV5ECFAXVPHHSK3YS5LQI3LQNTGYD3DNCHKT74VGRSA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.102365507 (238h9m9.905724671s) [noise=true , address=185.167.204.206:4041 , piece_id=JYD734S7X4QLARQFX44G24S7YXDUX3GWJ4QOY5NHD2NY5XMLVVOA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.103504275 (238h9m9.90686345s) [noise=true , address=89.149.201.147:4008 , piece_id=HY4JE6TBQ3AAXVGQNXVKIMX6MNU45UMLPK2YEH4QXC4OBKPNEWNQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.106507437 (238h9m9.909866612s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.145089732 (238h9m9.948448907s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.473005969 (238h9m10.276365143s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:13.403656159 (238h9m14.207015334s) [noise=true , address=185.167.204.206:4059 , piece_id=RYP2J5M7ERW64IPIDS4YC6XSMXJJYFY2FIOHFFVJ4FMEZMFC45PQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:14.229596272 (238h9m15.032955449s) [noise=true , address=185.167.204.206:4071 , piece_id=S23HGNGGEWNCQP2TMJ3CH2HL4TQWMEYVDFHCYKFAHXBDAY4X6ILA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:17.197218248 (238h9m18.000577422s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:50.077929419 (238h9m50.881288595s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:26:08.034087443 (238h10m8.837446626s) [noise=true , address=89.149.201.147:4016 , piece_id=DRJK3ZPIZKZ44E6RPX6FGQZWPNMA7VMAIP6JXCB7IUQQ7QC5F7BQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034276836 (238h10m8.837636013s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 159.223.143.39:60508->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034741100 (238h10m8.83810028s) [noise=true , address=185.167.204.205:4017 , piece_id=CV6LFILYHABDCW25TJJ7636AOKZ36JVLJJYXBNR7V4TZW4SJ7MUA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: read tcp 159.223.143.39:58588->185.167.204.205:4017: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034770401 (238h10m8.838129567s) [seg_pos={0 0}, upload=247902]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 18:26:08.139631321 (238h10m8.942990495s) [upload=247902]: Done uploading segments...
Feb 27 18:26:08.139695418 (238h10m8.943054606s) [upload=247902]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.138504853s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
13.948785ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin Accept-Ranges=bytes Server=Storj X-Amz-Request-Id=18982DE9469181DF X-Request-Id=hsZc3MyjMPV X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:28:53 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/31b32532b1d798aa
FAIL (2026-02-27 18:28:53 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DE9469181DF</RequestId><HostId></HostId></Error>"
Duration: 13.970784ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [................................X......................X....] duration:
2.097953462s trace:
https://jaeger.dev.storj.io/trace/7aa57c119569a500
FAIL (2026-02-27 18:28:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
KCFQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408102653 (238h9m9.211461838s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408567070 (238h9m9.211926247s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408707192 (238h9m9.212066378s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408968070 (238h9m9.212327244s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409367498 (238h9m9.212726677s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409377018 (238h9m9.212736185s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410004538 (238h9m9.213363725s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410656413 (238h9m9.214015608s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411331444 (238h9m9.214690672s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411885321 (238h9m9.215244506s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412446978 (238h9m9.21580616s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412960206 (238h9m9.216319389s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.833976270 (238h9m9.637335445s) [upload=247902]: Next returned nil segment
Feb 27 18:25:08.833993781 (238h9m9.637352963s) [upload=247902]: Waiting for error group managing segments...
Feb 27 18:25:09.002997616 (238h9m9.806356791s) [noise=true , address=81.171.14.70:4040 , piece_id=2MPY2M6YD4A35CALBOQBGARWAVYBWSALZU2P4JQNVTVU462KXOSQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.003988167 (238h9m9.807347334s) [noise=true , address=81.171.14.70:4037 , piece_id=O6CQWKOJPHOHZAO7PMTX74XFWNPWIDDDUE2S4NPO67A64TRGVLDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.007807120 (238h9m9.811166324s) [noise=true , address=81.171.14.70:4044 , piece_id=SX5RETWJXQ37MNWETTT3ISLFDEQMJ4LGWEWLA4SELKUG6L447YZQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.019462938 (238h9m9.822822115s) [noise=true , address=81.171.14.70:4039 , piece_id=VV7S6QSQSAMDYP2J4TWHDJNIGMTPEHG7N4737DETH4SSYKK3FKNA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.031013955 (238h9m9.834373141s) [noise=true , address=185.167.204.206:4050 , piece_id=NNXAXJUAW34ZSFMQZHSMONU2KSGJ4DNNNLGA3VCV7Q7JE4OXYUAA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.039442786 (238h9m9.84280196s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.050960211 (238h9m9.854319391s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.067324682 (238h9m9.870683857s) [noise=true , address=185.167.204.206:4067 , piece_id=LMTR6XVP77QRHIFJFPK2CCNTLFVKGCNFXBN2Z4AQQTE7YTJ2PDVA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.070489496 (238h9m9.87384866s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.071657801 (238h9m9.87501698s) [noise=true , address=185.167.204.206:4068 , piece_id=DRL3HHGWZQGLK5RWPTJFIOUERAFO3OH3FWGWCVCPLN3JAR7K7QPA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.072680314 (238h9m9.876039488s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.074685847 (238h9m9.878045033s) [noise=true , address=185.167.204.205:4008 , piece_id=CDKH6EYARWHKZIMNICX46Z3JSGBETOJEPGVGGESJ7LZZYGGBKKMQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.089811558 (238h9m9.893170735s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.098264180 (238h9m9.901623357s) [noise=true , address=185.167.204.206:4053 , piece_id=GOQ52CA2J4GSR7A5DXDDDMEDVHODA4FE5A6A2I3GMDSICFLSKCFQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101019345 (238h9m9.904378509s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101369094 (238h9m9.904728292s) [noise=true , address=185.167.204.205:4002 , piece_id=2DSKXMEVZMV5ECFAXVPHHSK3YS5LQI3LQNTGYD3DNCHKT74VGRSA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.102365507 (238h9m9.905724671s) [noise=true , address=185.167.204.206:4041 , piece_id=JYD734S7X4QLARQFX44G24S7YXDUX3GWJ4QOY5NHD2NY5XMLVVOA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.103504275 (238h9m9.90686345s) [noise=true , address=89.149.201.147:4008 , piece_id=HY4JE6TBQ3AAXVGQNXVKIMX6MNU45UMLPK2YEH4QXC4OBKPNEWNQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.106507437 (238h9m9.909866612s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.145089732 (238h9m9.948448907s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.473005969 (238h9m10.276365143s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:13.403656159 (238h9m14.207015334s) [noise=true , address=185.167.204.206:4059 , piece_id=RYP2J5M7ERW64IPIDS4YC6XSMXJJYFY2FIOHFFVJ4FMEZMFC45PQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:14.229596272 (238h9m15.032955449s) [noise=true , address=185.167.204.206:4071 , piece_id=S23HGNGGEWNCQP2TMJ3CH2HL4TQWMEYVDFHCYKFAHXBDAY4X6ILA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:17.197218248 (238h9m18.000577422s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:50.077929419 (238h9m50.881288595s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:26:08.034087443 (238h10m8.837446626s) [noise=true , address=89.149.201.147:4016 , piece_id=DRJK3ZPIZKZ44E6RPX6FGQZWPNMA7VMAIP6JXCB7IUQQ7QC5F7BQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034276836 (238h10m8.837636013s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 159.223.143.39:60508->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034741100 (238h10m8.83810028s) [noise=true , address=185.167.204.205:4017 , piece_id=CV6LFILYHABDCW25TJJ7636AOKZ36JVLJJYXBNR7V4TZW4SJ7MUA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: read tcp 159.223.143.39:58588->185.167.204.205:4017: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034770401 (238h10m8.838129567s) [seg_pos={0 0}, upload=247902]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 18:26:08.139631321 (238h10m8.942990495s) [upload=247902]: Done uploading segments...
Feb 27 18:26:08.139695418 (238h10m8.943054606s) [upload=247902]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.097978596s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.543461ms remote:
"tcp:136.0.77.3:443" response headers:
Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Content-Type=application/xml X-Amz-Request-Id=18982DE2F604E926 X-Request-Id=RLyFB7BUFZq Date=Fri, 27 Feb 2026 18:28:26 GMT Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/6d8874d4d68873a6
FAIL (2026-02-27 18:28:26 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DE2F604E926</RequestId><HostId></HostId></Error>"
Duration: 10.574595ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
29.509355ms remote:
"tcp:45.38.24.2:443" response headers:
Vary=Origin X-Amz-Request-Id=18982DE2158A5287 X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=284 X-Request-Id=PHbAqn2nwh2 Date=Fri, 27 Feb 2026 18:28:22 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/4f63a3f048c95aa5
FAIL (2026-02-27 18:28:22 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DE2158A5287</RequestId><HostId></HostId></Error>"
Duration: 29.528443ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.................................X......................X...] duration:
2.565915096s trace:
https://jaeger.dev.storj.io/trace/7aa57c119569a500
FAIL (2026-02-27 18:28:11 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
KCFQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408102653 (238h9m9.211461838s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408567070 (238h9m9.211926247s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408707192 (238h9m9.212066378s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408968070 (238h9m9.212327244s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409367498 (238h9m9.212726677s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409377018 (238h9m9.212736185s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410004538 (238h9m9.213363725s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410656413 (238h9m9.214015608s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411331444 (238h9m9.214690672s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411885321 (238h9m9.215244506s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412446978 (238h9m9.21580616s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412960206 (238h9m9.216319389s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.833976270 (238h9m9.637335445s) [upload=247902]: Next returned nil segment
Feb 27 18:25:08.833993781 (238h9m9.637352963s) [upload=247902]: Waiting for error group managing segments...
Feb 27 18:25:09.002997616 (238h9m9.806356791s) [noise=true , address=81.171.14.70:4040 , piece_id=2MPY2M6YD4A35CALBOQBGARWAVYBWSALZU2P4JQNVTVU462KXOSQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.003988167 (238h9m9.807347334s) [noise=true , address=81.171.14.70:4037 , piece_id=O6CQWKOJPHOHZAO7PMTX74XFWNPWIDDDUE2S4NPO67A64TRGVLDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.007807120 (238h9m9.811166324s) [noise=true , address=81.171.14.70:4044 , piece_id=SX5RETWJXQ37MNWETTT3ISLFDEQMJ4LGWEWLA4SELKUG6L447YZQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.019462938 (238h9m9.822822115s) [noise=true , address=81.171.14.70:4039 , piece_id=VV7S6QSQSAMDYP2J4TWHDJNIGMTPEHG7N4737DETH4SSYKK3FKNA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.031013955 (238h9m9.834373141s) [noise=true , address=185.167.204.206:4050 , piece_id=NNXAXJUAW34ZSFMQZHSMONU2KSGJ4DNNNLGA3VCV7Q7JE4OXYUAA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.039442786 (238h9m9.84280196s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.050960211 (238h9m9.854319391s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.067324682 (238h9m9.870683857s) [noise=true , address=185.167.204.206:4067 , piece_id=LMTR6XVP77QRHIFJFPK2CCNTLFVKGCNFXBN2Z4AQQTE7YTJ2PDVA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.070489496 (238h9m9.87384866s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.071657801 (238h9m9.87501698s) [noise=true , address=185.167.204.206:4068 , piece_id=DRL3HHGWZQGLK5RWPTJFIOUERAFO3OH3FWGWCVCPLN3JAR7K7QPA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.072680314 (238h9m9.876039488s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.074685847 (238h9m9.878045033s) [noise=true , address=185.167.204.205:4008 , piece_id=CDKH6EYARWHKZIMNICX46Z3JSGBETOJEPGVGGESJ7LZZYGGBKKMQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.089811558 (238h9m9.893170735s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.098264180 (238h9m9.901623357s) [noise=true , address=185.167.204.206:4053 , piece_id=GOQ52CA2J4GSR7A5DXDDDMEDVHODA4FE5A6A2I3GMDSICFLSKCFQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101019345 (238h9m9.904378509s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101369094 (238h9m9.904728292s) [noise=true , address=185.167.204.205:4002 , piece_id=2DSKXMEVZMV5ECFAXVPHHSK3YS5LQI3LQNTGYD3DNCHKT74VGRSA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.102365507 (238h9m9.905724671s) [noise=true , address=185.167.204.206:4041 , piece_id=JYD734S7X4QLARQFX44G24S7YXDUX3GWJ4QOY5NHD2NY5XMLVVOA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.103504275 (238h9m9.90686345s) [noise=true , address=89.149.201.147:4008 , piece_id=HY4JE6TBQ3AAXVGQNXVKIMX6MNU45UMLPK2YEH4QXC4OBKPNEWNQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.106507437 (238h9m9.909866612s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.145089732 (238h9m9.948448907s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.473005969 (238h9m10.276365143s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:13.403656159 (238h9m14.207015334s) [noise=true , address=185.167.204.206:4059 , piece_id=RYP2J5M7ERW64IPIDS4YC6XSMXJJYFY2FIOHFFVJ4FMEZMFC45PQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:14.229596272 (238h9m15.032955449s) [noise=true , address=185.167.204.206:4071 , piece_id=S23HGNGGEWNCQP2TMJ3CH2HL4TQWMEYVDFHCYKFAHXBDAY4X6ILA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:17.197218248 (238h9m18.000577422s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:50.077929419 (238h9m50.881288595s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:26:08.034087443 (238h10m8.837446626s) [noise=true , address=89.149.201.147:4016 , piece_id=DRJK3ZPIZKZ44E6RPX6FGQZWPNMA7VMAIP6JXCB7IUQQ7QC5F7BQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034276836 (238h10m8.837636013s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 159.223.143.39:60508->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034741100 (238h10m8.83810028s) [noise=true , address=185.167.204.205:4017 , piece_id=CV6LFILYHABDCW25TJJ7636AOKZ36JVLJJYXBNR7V4TZW4SJ7MUA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: read tcp 159.223.143.39:58588->185.167.204.205:4017: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034770401 (238h10m8.838129567s) [seg_pos={0 0}, upload=247902]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 18:26:08.139631321 (238h10m8.942990495s) [upload=247902]: Done uploading segments...
Feb 27 18:26:08.139695418 (238h10m8.943054606s) [upload=247902]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.565960013s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
27.970867ms remote:
"tcp:45.38.24.2:443" response headers:
X-Request-Id=dj4sc4vQDFh Date=Fri, 27 Feb 2026 18:28:07 GMT Accept-Ranges=bytes Server=Storj Vary=Origin X-Amz-Request-Id=18982DDE8BEB05B1 X-Xss-Protection=1; mode=block Content-Length=284 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/738b91886f315836
FAIL (2026-02-27 18:28:07 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DDE8BEB05B1</RequestId><HostId></HostId></Error>"
Duration: 27.994455ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
15.361162ms remote:
"tcp:136.0.77.3:443" response headers:
Server=Storj Vary=Origin Date=Fri, 27 Feb 2026 18:28:06 GMT Content-Type=application/xml X-Amz-Request-Id=18982DDE62802BB2 X-Request-Id=PFnMX7bj9Hd X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/6936bea93d68f7e5
FAIL (2026-02-27 18:28:06 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DDE62802BB2</RequestId><HostId></HostId></Error>"
Duration: 15.383894ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
15.410757ms remote:
"tcp:136.0.77.3:443" response headers:
X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:27:42 GMT Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml Vary=Origin Content-Security-Policy=block-all-mixed-content Server=Storj X-Amz-Request-Id=18982DD8C6FCBA39 X-Request-Id=LYWnRsF18Ad target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/4c6e79d409ae9746
FAIL (2026-02-27 18:27:42 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DD8C6FCBA39</RequestId><HostId></HostId></Error>"
Duration: 15.626348ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
29.191943ms remote:
"tcp:45.38.24.2:443" response headers:
X-Amz-Request-Id=18982DD4EEFAD36F Date=Fri, 27 Feb 2026 18:27:25 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content X-Request-Id=Be3XyYm5vpP X-Xss-Protection=1; mode=block Content-Length=284 Content-Type=application/xml Server=Storj Vary=Origin target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/6b581b7496f862b7
FAIL (2026-02-27 18:27:25 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DD4EEFAD36F</RequestId><HostId></HostId></Error>"
Duration: 29.215184ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.179936ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Content-Length=274 Content-Type=application/xml X-Amz-Request-Id=18982DD1BD897AD6 X-Request-Id=DNzZ6pAwnYe Date=Fri, 27 Feb 2026 18:27:12 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/636f498da56e193c
FAIL (2026-02-27 18:27:12 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DD1BD897AD6</RequestId><HostId></HostId></Error>"
Duration: 10.206702ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.492278ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Accept-Ranges=bytes Content-Length=274 Vary=Origin X-Amz-Request-Id=18982DD1CBDB0E0A X-Request-Id=7FXe2fTWBYQ X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:27:12 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/3f8853e62450ca1e
FAIL (2026-02-27 18:27:12 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DD1CBDB0E0A</RequestId><HostId></HostId></Error>"
Duration: 11.54422ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
14.591945ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin Date=Fri, 27 Feb 2026 18:27:11 GMT X-Amz-Request-Id=18982DD1A5D4B021 X-Request-Id=JkqqcYjgnV X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/6ad89e8cd97fb3be
FAIL (2026-02-27 18:27:11 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DD1A5D4B021</RequestId><HostId></HostId></Error>"
Duration: 14.617223ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..................................X......................X..] duration:
2.10286821s trace:
https://jaeger.dev.storj.io/trace/7aa57c119569a500
FAIL (2026-02-27 18:27:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
KCFQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408102653 (238h9m9.211461838s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408567070 (238h9m9.211926247s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408707192 (238h9m9.212066378s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408968070 (238h9m9.212327244s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409367498 (238h9m9.212726677s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409377018 (238h9m9.212736185s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410004538 (238h9m9.213363725s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410656413 (238h9m9.214015608s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411331444 (238h9m9.214690672s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411885321 (238h9m9.215244506s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412446978 (238h9m9.21580616s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412960206 (238h9m9.216319389s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.833976270 (238h9m9.637335445s) [upload=247902]: Next returned nil segment
Feb 27 18:25:08.833993781 (238h9m9.637352963s) [upload=247902]: Waiting for error group managing segments...
Feb 27 18:25:09.002997616 (238h9m9.806356791s) [noise=true , address=81.171.14.70:4040 , piece_id=2MPY2M6YD4A35CALBOQBGARWAVYBWSALZU2P4JQNVTVU462KXOSQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.003988167 (238h9m9.807347334s) [noise=true , address=81.171.14.70:4037 , piece_id=O6CQWKOJPHOHZAO7PMTX74XFWNPWIDDDUE2S4NPO67A64TRGVLDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.007807120 (238h9m9.811166324s) [noise=true , address=81.171.14.70:4044 , piece_id=SX5RETWJXQ37MNWETTT3ISLFDEQMJ4LGWEWLA4SELKUG6L447YZQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.019462938 (238h9m9.822822115s) [noise=true , address=81.171.14.70:4039 , piece_id=VV7S6QSQSAMDYP2J4TWHDJNIGMTPEHG7N4737DETH4SSYKK3FKNA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.031013955 (238h9m9.834373141s) [noise=true , address=185.167.204.206:4050 , piece_id=NNXAXJUAW34ZSFMQZHSMONU2KSGJ4DNNNLGA3VCV7Q7JE4OXYUAA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.039442786 (238h9m9.84280196s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.050960211 (238h9m9.854319391s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.067324682 (238h9m9.870683857s) [noise=true , address=185.167.204.206:4067 , piece_id=LMTR6XVP77QRHIFJFPK2CCNTLFVKGCNFXBN2Z4AQQTE7YTJ2PDVA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.070489496 (238h9m9.87384866s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.071657801 (238h9m9.87501698s) [noise=true , address=185.167.204.206:4068 , piece_id=DRL3HHGWZQGLK5RWPTJFIOUERAFO3OH3FWGWCVCPLN3JAR7K7QPA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.072680314 (238h9m9.876039488s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.074685847 (238h9m9.878045033s) [noise=true , address=185.167.204.205:4008 , piece_id=CDKH6EYARWHKZIMNICX46Z3JSGBETOJEPGVGGESJ7LZZYGGBKKMQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.089811558 (238h9m9.893170735s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.098264180 (238h9m9.901623357s) [noise=true , address=185.167.204.206:4053 , piece_id=GOQ52CA2J4GSR7A5DXDDDMEDVHODA4FE5A6A2I3GMDSICFLSKCFQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101019345 (238h9m9.904378509s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101369094 (238h9m9.904728292s) [noise=true , address=185.167.204.205:4002 , piece_id=2DSKXMEVZMV5ECFAXVPHHSK3YS5LQI3LQNTGYD3DNCHKT74VGRSA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.102365507 (238h9m9.905724671s) [noise=true , address=185.167.204.206:4041 , piece_id=JYD734S7X4QLARQFX44G24S7YXDUX3GWJ4QOY5NHD2NY5XMLVVOA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.103504275 (238h9m9.90686345s) [noise=true , address=89.149.201.147:4008 , piece_id=HY4JE6TBQ3AAXVGQNXVKIMX6MNU45UMLPK2YEH4QXC4OBKPNEWNQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.106507437 (238h9m9.909866612s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.145089732 (238h9m9.948448907s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.473005969 (238h9m10.276365143s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:13.403656159 (238h9m14.207015334s) [noise=true , address=185.167.204.206:4059 , piece_id=RYP2J5M7ERW64IPIDS4YC6XSMXJJYFY2FIOHFFVJ4FMEZMFC45PQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:14.229596272 (238h9m15.032955449s) [noise=true , address=185.167.204.206:4071 , piece_id=S23HGNGGEWNCQP2TMJ3CH2HL4TQWMEYVDFHCYKFAHXBDAY4X6ILA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:17.197218248 (238h9m18.000577422s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:50.077929419 (238h9m50.881288595s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:26:08.034087443 (238h10m8.837446626s) [noise=true , address=89.149.201.147:4016 , piece_id=DRJK3ZPIZKZ44E6RPX6FGQZWPNMA7VMAIP6JXCB7IUQQ7QC5F7BQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034276836 (238h10m8.837636013s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 159.223.143.39:60508->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034741100 (238h10m8.83810028s) [noise=true , address=185.167.204.205:4017 , piece_id=CV6LFILYHABDCW25TJJ7636AOKZ36JVLJJYXBNR7V4TZW4SJ7MUA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: read tcp 159.223.143.39:58588->185.167.204.205:4017: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034770401 (238h10m8.838129567s) [seg_pos={0 0}, upload=247902]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 18:26:08.139631321 (238h10m8.942990495s) [upload=247902]: Done uploading segments...
Feb 27 18:26:08.139695418 (238h10m8.943054606s) [upload=247902]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.102886923s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...................................X......................X.] duration:
49.796150718s trace:
https://jaeger.dev.storj.io/trace/7aa57c119569a500
FAIL (2026-02-27 18:26:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
KCFQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408102653 (238h9m9.211461838s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408567070 (238h9m9.211926247s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408707192 (238h9m9.212066378s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408968070 (238h9m9.212327244s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409367498 (238h9m9.212726677s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409377018 (238h9m9.212736185s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410004538 (238h9m9.213363725s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410656413 (238h9m9.214015608s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411331444 (238h9m9.214690672s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411885321 (238h9m9.215244506s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412446978 (238h9m9.21580616s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412960206 (238h9m9.216319389s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.833976270 (238h9m9.637335445s) [upload=247902]: Next returned nil segment
Feb 27 18:25:08.833993781 (238h9m9.637352963s) [upload=247902]: Waiting for error group managing segments...
Feb 27 18:25:09.002997616 (238h9m9.806356791s) [noise=true , address=81.171.14.70:4040 , piece_id=2MPY2M6YD4A35CALBOQBGARWAVYBWSALZU2P4JQNVTVU462KXOSQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.003988167 (238h9m9.807347334s) [noise=true , address=81.171.14.70:4037 , piece_id=O6CQWKOJPHOHZAO7PMTX74XFWNPWIDDDUE2S4NPO67A64TRGVLDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.007807120 (238h9m9.811166324s) [noise=true , address=81.171.14.70:4044 , piece_id=SX5RETWJXQ37MNWETTT3ISLFDEQMJ4LGWEWLA4SELKUG6L447YZQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.019462938 (238h9m9.822822115s) [noise=true , address=81.171.14.70:4039 , piece_id=VV7S6QSQSAMDYP2J4TWHDJNIGMTPEHG7N4737DETH4SSYKK3FKNA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.031013955 (238h9m9.834373141s) [noise=true , address=185.167.204.206:4050 , piece_id=NNXAXJUAW34ZSFMQZHSMONU2KSGJ4DNNNLGA3VCV7Q7JE4OXYUAA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.039442786 (238h9m9.84280196s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.050960211 (238h9m9.854319391s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.067324682 (238h9m9.870683857s) [noise=true , address=185.167.204.206:4067 , piece_id=LMTR6XVP77QRHIFJFPK2CCNTLFVKGCNFXBN2Z4AQQTE7YTJ2PDVA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.070489496 (238h9m9.87384866s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.071657801 (238h9m9.87501698s) [noise=true , address=185.167.204.206:4068 , piece_id=DRL3HHGWZQGLK5RWPTJFIOUERAFO3OH3FWGWCVCPLN3JAR7K7QPA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.072680314 (238h9m9.876039488s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.074685847 (238h9m9.878045033s) [noise=true , address=185.167.204.205:4008 , piece_id=CDKH6EYARWHKZIMNICX46Z3JSGBETOJEPGVGGESJ7LZZYGGBKKMQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.089811558 (238h9m9.893170735s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.098264180 (238h9m9.901623357s) [noise=true , address=185.167.204.206:4053 , piece_id=GOQ52CA2J4GSR7A5DXDDDMEDVHODA4FE5A6A2I3GMDSICFLSKCFQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101019345 (238h9m9.904378509s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101369094 (238h9m9.904728292s) [noise=true , address=185.167.204.205:4002 , piece_id=2DSKXMEVZMV5ECFAXVPHHSK3YS5LQI3LQNTGYD3DNCHKT74VGRSA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.102365507 (238h9m9.905724671s) [noise=true , address=185.167.204.206:4041 , piece_id=JYD734S7X4QLARQFX44G24S7YXDUX3GWJ4QOY5NHD2NY5XMLVVOA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.103504275 (238h9m9.90686345s) [noise=true , address=89.149.201.147:4008 , piece_id=HY4JE6TBQ3AAXVGQNXVKIMX6MNU45UMLPK2YEH4QXC4OBKPNEWNQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.106507437 (238h9m9.909866612s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.145089732 (238h9m9.948448907s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.473005969 (238h9m10.276365143s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:13.403656159 (238h9m14.207015334s) [noise=true , address=185.167.204.206:4059 , piece_id=RYP2J5M7ERW64IPIDS4YC6XSMXJJYFY2FIOHFFVJ4FMEZMFC45PQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:14.229596272 (238h9m15.032955449s) [noise=true , address=185.167.204.206:4071 , piece_id=S23HGNGGEWNCQP2TMJ3CH2HL4TQWMEYVDFHCYKFAHXBDAY4X6ILA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:17.197218248 (238h9m18.000577422s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:50.077929419 (238h9m50.881288595s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:26:08.034087443 (238h10m8.837446626s) [noise=true , address=89.149.201.147:4016 , piece_id=DRJK3ZPIZKZ44E6RPX6FGQZWPNMA7VMAIP6JXCB7IUQQ7QC5F7BQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034276836 (238h10m8.837636013s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 159.223.143.39:60508->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034741100 (238h10m8.83810028s) [noise=true , address=185.167.204.205:4017 , piece_id=CV6LFILYHABDCW25TJJ7636AOKZ36JVLJJYXBNR7V4TZW4SJ7MUA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: read tcp 159.223.143.39:58588->185.167.204.205:4017: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034770401 (238h10m8.838129567s) [seg_pos={0 0}, upload=247902]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 18:26:08.139631321 (238h10m8.942990495s) [upload=247902]: Done uploading segments...
Feb 27 18:26:08.139695418 (238h10m8.943054606s) [upload=247902]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 49.796174226s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
14.288274ms remote:
"tcp:136.0.77.3:443" response headers:
X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:26:44 GMT Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml X-Amz-Request-Id=18982DCB5FA44AC2 X-Request-Id=39krHP9bUPk target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/1e459f604f2d9645
FAIL (2026-02-27 18:26:44 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DCB5FA44AC2</RequestId><HostId></HostId></Error>"
Duration: 14.308358ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
29.891985ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Type=application/xml Vary=Origin Content-Length=284 Content-Security-Policy=block-all-mixed-content Server=Storj X-Amz-Request-Id=18982DC52B3FDFC0 X-Request-Id=L2WwBPXUjFA X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 18:26:18 GMT Accept-Ranges=bytes target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/2d23741f7dfc233b
FAIL (2026-02-27 18:26:18 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DC52B3FDFC0</RequestId><HostId></HostId></Error>"
Duration: 29.91616ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
14.802922ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Type=application/xml X-Amz-Request-Id=18982DC4312EEC3D Date=Fri, 27 Feb 2026 18:26:14 GMT Server=Storj Vary=Origin X-Request-Id=coXf2ipnZ3f X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/47141d9cb439c002
FAIL (2026-02-27 18:26:14 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982DC4312EEC3D</RequestId><HostId></HostId></Error>"
Duration: 14.829322ms
edge
gateway
vhost
bgp
eu1
|