Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.064124ms remote:
"tcp:136.0.77.3:443" response headers:
Date=Fri, 27 Feb 2026 19:08:39 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content Vary=Origin X-Amz-Request-Id=18983014D845CE0A Accept-Ranges=bytes Content-Type=application/xml Server=Storj X-Request-Id=8npkAyVKjxD X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/66568dc11a02e4c4
FAIL (2026-02-27 19:08:39 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18983014D845CE0A</RequestId><HostId></HostId></Error>"
Duration: 9.086745ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.77711ms remote:
"tcp:136.0.77.3:443" response headers:
X-Xss-Protection=1; mode=block Accept-Ranges=bytes Server=Storj X-Request-Id=gRQbcKZ7jhf Date=Fri, 27 Feb 2026 19:08:28 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18983012333927D6 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/54a8e0fc1f70ea70
FAIL (2026-02-27 19:08:28 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18983012333927D6</RequestId><HostId></HostId></Error>"
Duration: 9.798889ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.845106ms remote:
"tcp:136.0.77.3:443" response headers:
Server=Storj Vary=Origin Content-Length=274 X-Amz-Request-Id=18983010A6222675 X-Request-Id=eMrMDJzskm2 X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:08:21 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/21bdd009f45864d0
FAIL (2026-02-27 19:08:21 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18983010A6222675</RequestId><HostId></HostId></Error>"
Duration: 10.869339ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [X...................................................X.......] duration:
2.088735207s trace:
https://jaeger.dev.storj.io/trace/5ba7b9ab33770c8
FAIL (2026-02-27 19:08:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
ess=185.167.204.205:4020 , piece_id=BRCPVT4JBAXGNYY7N547DSMTC7L2IKGG7RHT7MU7ISL2NC3ZDBCQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.881920320 (238h47m29.685279497s) [noise=true , address=185.167.204.206:4046 , piece_id=6ZTW47UKRKWVYGJHOBHANVWZWE6PDHRLITMD5VUNCARVASHK47AQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882277613 (238h47m29.685636797s) [noise=true , address=81.171.14.70:4025 , piece_id=JAKRF6PR6HKFBA536E3EQQ3WCWKDTYUXXCWG4THF5AFMACWK5H5Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882304171 (238h47m29.685663353s) [noise=true , address=89.149.201.147:4001 , piece_id=ORKT2SIHXBZWT626NROJIJGONLRAI3YOFHAFDQ7JOOEQMHNDPAPQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882721732 (238h47m29.68608091s) [noise=true , address=81.171.14.70:4038 , piece_id=KKVAZ7XUADG7JYHVSIXKTHJDXKCUICAGRZSBZHCSPD36LW6OYT2Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.883170726 (238h47m29.686529903s) [noise=true , address=89.149.201.147:4017 , piece_id=EQXPHI4WOCMNHULZT3KP4RXHHCUVM77FGPPLOQOSBODSKAE4NVRQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.883675356 (238h47m29.687034539s) [noise=true , address=185.167.204.206:4040 , piece_id=NBIGXDP7K2G2DYUTEHCSML4X5QF6DWITXNOX6G6LJCWOLUZZJWHA, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.884309144 (238h47m29.687668328s) [noise=true , address=89.149.201.147:4023 , piece_id=NWUSBCFDTVH67AO5RFAUMRBP7I4BA6LM2SS46REMDANS5PULE33A, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.884827042 (238h47m29.68818621s) [noise=true , address=185.167.204.206:4041 , piece_id=PNMLNBEI5MCZSXIELZF22U7C5FBTJ3NKZVSMHHBZ4NHRO4CE2JMQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.887870537 (238h47m29.691229718s) [noise=true , address=89.149.201.147:4010 , piece_id=RY3V22SFDMDJL2SA67IVSIA3HL5ZXWFPDFEGKVTN3OJ6A3EY257Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.888962425 (238h47m29.692321668s) [noise=true , address=81.171.14.70:4030 , piece_id=RXX3L2JLRR63LJMPPHD4KIB4S3ETKXPJGZ7SO7YIVEPNX7J4SOUA, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.889823462 (238h47m29.693182639s) [noise=true , address=185.167.204.205:4016 , piece_id=TKSQKNTBVIDDBR3UPJLX7ZCNMFMEXYPXL7YJCQ4ZSPYG2H76PY3A, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:29.305942479 (238h47m30.109301644s) [upload=248562]: Next returned nil segment
Feb 27 19:03:29.305967164 (238h47m30.109326339s) [upload=248562]: Waiting for error group managing segments...
Feb 27 19:03:29.485292528 (238h47m30.288651714s) [noise=true , address=81.171.14.70:4032 , piece_id=GVGNNAVZFDEQPI6BT6ZRDOWEDEJGTI5CRWZQDQ7ROPCYV4WU2OMQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.488867142 (238h47m30.292226317s) [noise=true , address=81.171.14.70:4042 , piece_id=JK6BODBCGQGSFSFPS4DADNUW4SLFV2ERJPWT6IZVVGD5JNBQXKQA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.488939564 (238h47m30.292298726s) [noise=true , address=81.171.14.70:4038 , piece_id=KKVAZ7XUADG7JYHVSIXKTHJDXKCUICAGRZSBZHCSPD36LW6OYT2Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.493270337 (238h47m30.29662951s) [noise=true , address=81.171.14.70:4025 , piece_id=JAKRF6PR6HKFBA536E3EQQ3WCWKDTYUXXCWG4THF5AFMACWK5H5Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.494182345 (238h47m30.297541509s) [noise=true , address=81.171.14.70:4030 , piece_id=RXX3L2JLRR63LJMPPHD4KIB4S3ETKXPJGZ7SO7YIVEPNX7J4SOUA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.495569378 (238h47m30.298928546s) [noise=true , address=81.171.14.70:4034 , piece_id=TKUZL52SXKKF4KUVCSTYIMJTTBZZ5NZ7WYQSLH4Z4DMYQD2GLWNA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.497221478 (238h47m30.300580654s) [noise=true , address=185.167.204.205:4016 , piece_id=TKSQKNTBVIDDBR3UPJLX7ZCNMFMEXYPXL7YJCQ4ZSPYG2H76PY3A, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.498481050 (238h47m30.301840223s) [noise=true , address=81.171.14.70:4047 , piece_id=XSDESMXZSX2FPVCIM6X533ZVBQ2NRCKCUSJ7UBEOCG76RCVOFFQA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.499128725 (238h47m30.302487899s) [noise=true , address=81.171.14.70:4033 , piece_id=UTG5PYLTCAKPYWHBFIEHYPQGWX7V2CWPPZUS3IHHM6IWHG2RSYIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.500770132 (238h47m30.304129311s) [noise=true , address=185.167.204.205:4011 , piece_id=BGJAKT35K6HYKSBM75R7Y527P2WEJHTWDZNVVYZOKLZOD3GWQDTA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.520282724 (238h47m30.323641898s) [noise=true , address=89.149.201.147:4024 , piece_id=SPWYXRSYXBFJVRYRX7KUN2PPIKMOJAM5PEJJMDTU7MCO52BBOGIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.520458144 (238h47m30.323817325s) [noise=true , address=89.149.201.147:4017 , piece_id=EQXPHI4WOCMNHULZT3KP4RXHHCUVM77FGPPLOQOSBODSKAE4NVRQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.528608534 (238h47m30.331967713s) [noise=true , address=185.167.204.206:4040 , piece_id=NBIGXDP7K2G2DYUTEHCSML4X5QF6DWITXNOX6G6LJCWOLUZZJWHA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.530517514 (238h47m30.333876679s) [noise=true , address=81.171.14.70:4029 , piece_id=P4PKJ6ZBWW5XJQ3ABVC5KDGVROK6TWZFEYOLJIMVO4GH4P27NY5Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.540229723 (238h47m30.343588896s) [noise=true , address=185.167.204.206:4039 , piece_id=B4BO7RY4U3QHZKNAAGUKFVLO3F7SCHPYM4TNUXBYM3UWIFC6RSIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.549193658 (238h47m30.352552843s) [noise=true , address=185.167.204.206:4037 , piece_id=4QRWKI66B2SVG2OWNQ5MQMQO33ZLWEV6OABDFMQWDNJVHWMRTJFQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.573203429 (238h47m30.376562603s) [noise=true , address=185.167.204.205:4007 , piece_id=JSFJFE6J7T3OYZUCU6OSGBTOGTDKXWKUX2GKZBZJPC4XMVUHI2RA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.578612555 (238h47m30.381971729s) [noise=true , address=185.167.204.206:4072 , piece_id=NWCN7VZ7C77J32GXW7MUFMACI3V45J7YQPRWIVNICO5E6N6Q5DBA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.578869362 (238h47m30.382228531s) [noise=true , address=185.167.204.206:4041 , piece_id=PNMLNBEI5MCZSXIELZF22U7C5FBTJ3NKZVSMHHBZ4NHRO4CE2JMQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.585453146 (238h47m30.388812327s) [noise=true , address=89.149.201.147:4004 , piece_id=R6KLLIMXVNYBDWH2W5KCOBWCQYF7RO47TFQKPHOR3UTYY446ATQQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.585510255 (238h47m30.388869432s) [noise=true , address=185.167.204.205:4009 , piece_id=UW66SRSI27LFSDXBPXZVA4SOIXQSEDIQAGCCPNFOIDYC6CO5DNXA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.607509471 (238h47m30.41086865s) [noise=true , address=185.167.204.206:4053 , piece_id=7CEFYGFPOKLIDAZWBEPBJ43O4OVYYUF66WABXWM7RADT4OBQ4AWQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.622982646 (238h47m30.426341819s) [noise=true , address=185.167.204.206:4046 , piece_id=6ZTW47UKRKWVYGJHOBHANVWZWE6PDHRLITMD5VUNCARVASHK47AQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.635158349 (238h47m30.438517522s) [noise=true , address=185.167.204.205:4020 , piece_id=BRCPVT4JBAXGNYY7N547DSMTC7L2IKGG7RHT7MU7ISL2NC3ZDBCQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:35.525495067 (238h47m36.328854242s) [noise=true , address=185.167.204.206:4044 , piece_id=2XMMX3NSARIQBL6SHBUTLTCTN7VC5HNHW5B5VGX4SBKNJTUDBSNA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:04:28.508369911 (238h48m29.311729091s) [noise=true , address=89.149.201.147:4010 , piece_id=RY3V22SFDMDJL2SA67IVSIA3HL5ZXWFPDFEGKVTN3OJ6A3EY257Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 159.223.143.39:56278->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 19:04:28.508664395 (238h48m29.312023562s) [noise=true , address=89.149.201.147:4001 , piece_id=ORKT2SIHXBZWT626NROJIJGONLRAI3YOFHAFDQ7JOOEQMHNDPAPQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8, address:89.149.201.147:4001): protocol: read tcp 159.223.143.39:49320->89.149.201.147:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 19:04:28.508923513 (238h48m29.312282696s) [noise=true , address=89.149.201.147:4023 , piece_id=NWUSBCFDTVH67AO5RFAUMRBP7I4BA6LM2SS46REMDANS5PULE33A, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv, address:89.149.201.147:4023): protocol: read tcp 159.223.143.39:48854->89.149.201.147:4023: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Feb 27 19:04:28.508961365 (238h48m29.312320533s) [seg_pos={0 0}, upload=248562]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 19:04:28.611109748 (238h48m29.414468923s) [upload=248562]: Done uploading segments...
Feb 27 19:04:28.611159571 (238h48m29.414518748s) [upload=248562]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.088766341s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
13.921916ms remote:
"tcp:136.0.77.3:443" response headers:
Date=Fri, 27 Feb 2026 19:08:05 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content Accept-Ranges=bytes Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=1898300CD7AEB29B X-Request-Id=bYxMqYcNJm5 X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/5108190485a21f27
FAIL (2026-02-27 19:08:05 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898300CD7AEB29B</RequestId><HostId></HostId></Error>"
Duration: 13.939231ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.275738ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Type=application/xml Vary=Origin Date=Fri, 27 Feb 2026 19:07:57 GMT Accept-Ranges=bytes Content-Length=274 Server=Storj X-Amz-Request-Id=1898300B03F0F73F X-Request-Id=EvydNWiusJM X-Xss-Protection=1; mode=block Content-Security-Policy=block-all-mixed-content target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/3faed48e273fb41
FAIL (2026-02-27 19:07:57 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898300B03F0F73F</RequestId><HostId></HostId></Error>"
Duration: 11.304973ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.398155ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=18983009C16BB5B9 X-Request-Id=HCL3FduMDp7 X-Xss-Protection=1; mode=block Content-Security-Policy=block-all-mixed-content Date=Fri, 27 Feb 2026 19:07:51 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/618e2a5a711649c
FAIL (2026-02-27 19:07:51 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18983009C16BB5B9</RequestId><HostId></HostId></Error>"
Duration: 11.41665ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.X...................................................X......] duration:
2.03987728s trace:
https://jaeger.dev.storj.io/trace/5ba7b9ab33770c8
FAIL (2026-02-27 19:07:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
ess=185.167.204.205:4020 , piece_id=BRCPVT4JBAXGNYY7N547DSMTC7L2IKGG7RHT7MU7ISL2NC3ZDBCQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.881920320 (238h47m29.685279497s) [noise=true , address=185.167.204.206:4046 , piece_id=6ZTW47UKRKWVYGJHOBHANVWZWE6PDHRLITMD5VUNCARVASHK47AQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882277613 (238h47m29.685636797s) [noise=true , address=81.171.14.70:4025 , piece_id=JAKRF6PR6HKFBA536E3EQQ3WCWKDTYUXXCWG4THF5AFMACWK5H5Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882304171 (238h47m29.685663353s) [noise=true , address=89.149.201.147:4001 , piece_id=ORKT2SIHXBZWT626NROJIJGONLRAI3YOFHAFDQ7JOOEQMHNDPAPQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882721732 (238h47m29.68608091s) [noise=true , address=81.171.14.70:4038 , piece_id=KKVAZ7XUADG7JYHVSIXKTHJDXKCUICAGRZSBZHCSPD36LW6OYT2Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.883170726 (238h47m29.686529903s) [noise=true , address=89.149.201.147:4017 , piece_id=EQXPHI4WOCMNHULZT3KP4RXHHCUVM77FGPPLOQOSBODSKAE4NVRQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.883675356 (238h47m29.687034539s) [noise=true , address=185.167.204.206:4040 , piece_id=NBIGXDP7K2G2DYUTEHCSML4X5QF6DWITXNOX6G6LJCWOLUZZJWHA, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.884309144 (238h47m29.687668328s) [noise=true , address=89.149.201.147:4023 , piece_id=NWUSBCFDTVH67AO5RFAUMRBP7I4BA6LM2SS46REMDANS5PULE33A, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.884827042 (238h47m29.68818621s) [noise=true , address=185.167.204.206:4041 , piece_id=PNMLNBEI5MCZSXIELZF22U7C5FBTJ3NKZVSMHHBZ4NHRO4CE2JMQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.887870537 (238h47m29.691229718s) [noise=true , address=89.149.201.147:4010 , piece_id=RY3V22SFDMDJL2SA67IVSIA3HL5ZXWFPDFEGKVTN3OJ6A3EY257Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.888962425 (238h47m29.692321668s) [noise=true , address=81.171.14.70:4030 , piece_id=RXX3L2JLRR63LJMPPHD4KIB4S3ETKXPJGZ7SO7YIVEPNX7J4SOUA, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.889823462 (238h47m29.693182639s) [noise=true , address=185.167.204.205:4016 , piece_id=TKSQKNTBVIDDBR3UPJLX7ZCNMFMEXYPXL7YJCQ4ZSPYG2H76PY3A, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:29.305942479 (238h47m30.109301644s) [upload=248562]: Next returned nil segment
Feb 27 19:03:29.305967164 (238h47m30.109326339s) [upload=248562]: Waiting for error group managing segments...
Feb 27 19:03:29.485292528 (238h47m30.288651714s) [noise=true , address=81.171.14.70:4032 , piece_id=GVGNNAVZFDEQPI6BT6ZRDOWEDEJGTI5CRWZQDQ7ROPCYV4WU2OMQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.488867142 (238h47m30.292226317s) [noise=true , address=81.171.14.70:4042 , piece_id=JK6BODBCGQGSFSFPS4DADNUW4SLFV2ERJPWT6IZVVGD5JNBQXKQA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.488939564 (238h47m30.292298726s) [noise=true , address=81.171.14.70:4038 , piece_id=KKVAZ7XUADG7JYHVSIXKTHJDXKCUICAGRZSBZHCSPD36LW6OYT2Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.493270337 (238h47m30.29662951s) [noise=true , address=81.171.14.70:4025 , piece_id=JAKRF6PR6HKFBA536E3EQQ3WCWKDTYUXXCWG4THF5AFMACWK5H5Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.494182345 (238h47m30.297541509s) [noise=true , address=81.171.14.70:4030 , piece_id=RXX3L2JLRR63LJMPPHD4KIB4S3ETKXPJGZ7SO7YIVEPNX7J4SOUA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.495569378 (238h47m30.298928546s) [noise=true , address=81.171.14.70:4034 , piece_id=TKUZL52SXKKF4KUVCSTYIMJTTBZZ5NZ7WYQSLH4Z4DMYQD2GLWNA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.497221478 (238h47m30.300580654s) [noise=true , address=185.167.204.205:4016 , piece_id=TKSQKNTBVIDDBR3UPJLX7ZCNMFMEXYPXL7YJCQ4ZSPYG2H76PY3A, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.498481050 (238h47m30.301840223s) [noise=true , address=81.171.14.70:4047 , piece_id=XSDESMXZSX2FPVCIM6X533ZVBQ2NRCKCUSJ7UBEOCG76RCVOFFQA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.499128725 (238h47m30.302487899s) [noise=true , address=81.171.14.70:4033 , piece_id=UTG5PYLTCAKPYWHBFIEHYPQGWX7V2CWPPZUS3IHHM6IWHG2RSYIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.500770132 (238h47m30.304129311s) [noise=true , address=185.167.204.205:4011 , piece_id=BGJAKT35K6HYKSBM75R7Y527P2WEJHTWDZNVVYZOKLZOD3GWQDTA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.520282724 (238h47m30.323641898s) [noise=true , address=89.149.201.147:4024 , piece_id=SPWYXRSYXBFJVRYRX7KUN2PPIKMOJAM5PEJJMDTU7MCO52BBOGIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.520458144 (238h47m30.323817325s) [noise=true , address=89.149.201.147:4017 , piece_id=EQXPHI4WOCMNHULZT3KP4RXHHCUVM77FGPPLOQOSBODSKAE4NVRQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.528608534 (238h47m30.331967713s) [noise=true , address=185.167.204.206:4040 , piece_id=NBIGXDP7K2G2DYUTEHCSML4X5QF6DWITXNOX6G6LJCWOLUZZJWHA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.530517514 (238h47m30.333876679s) [noise=true , address=81.171.14.70:4029 , piece_id=P4PKJ6ZBWW5XJQ3ABVC5KDGVROK6TWZFEYOLJIMVO4GH4P27NY5Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.540229723 (238h47m30.343588896s) [noise=true , address=185.167.204.206:4039 , piece_id=B4BO7RY4U3QHZKNAAGUKFVLO3F7SCHPYM4TNUXBYM3UWIFC6RSIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.549193658 (238h47m30.352552843s) [noise=true , address=185.167.204.206:4037 , piece_id=4QRWKI66B2SVG2OWNQ5MQMQO33ZLWEV6OABDFMQWDNJVHWMRTJFQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.573203429 (238h47m30.376562603s) [noise=true , address=185.167.204.205:4007 , piece_id=JSFJFE6J7T3OYZUCU6OSGBTOGTDKXWKUX2GKZBZJPC4XMVUHI2RA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.578612555 (238h47m30.381971729s) [noise=true , address=185.167.204.206:4072 , piece_id=NWCN7VZ7C77J32GXW7MUFMACI3V45J7YQPRWIVNICO5E6N6Q5DBA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.578869362 (238h47m30.382228531s) [noise=true , address=185.167.204.206:4041 , piece_id=PNMLNBEI5MCZSXIELZF22U7C5FBTJ3NKZVSMHHBZ4NHRO4CE2JMQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.585453146 (238h47m30.388812327s) [noise=true , address=89.149.201.147:4004 , piece_id=R6KLLIMXVNYBDWH2W5KCOBWCQYF7RO47TFQKPHOR3UTYY446ATQQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.585510255 (238h47m30.388869432s) [noise=true , address=185.167.204.205:4009 , piece_id=UW66SRSI27LFSDXBPXZVA4SOIXQSEDIQAGCCPNFOIDYC6CO5DNXA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.607509471 (238h47m30.41086865s) [noise=true , address=185.167.204.206:4053 , piece_id=7CEFYGFPOKLIDAZWBEPBJ43O4OVYYUF66WABXWM7RADT4OBQ4AWQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.622982646 (238h47m30.426341819s) [noise=true , address=185.167.204.206:4046 , piece_id=6ZTW47UKRKWVYGJHOBHANVWZWE6PDHRLITMD5VUNCARVASHK47AQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.635158349 (238h47m30.438517522s) [noise=true , address=185.167.204.205:4020 , piece_id=BRCPVT4JBAXGNYY7N547DSMTC7L2IKGG7RHT7MU7ISL2NC3ZDBCQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:35.525495067 (238h47m36.328854242s) [noise=true , address=185.167.204.206:4044 , piece_id=2XMMX3NSARIQBL6SHBUTLTCTN7VC5HNHW5B5VGX4SBKNJTUDBSNA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:04:28.508369911 (238h48m29.311729091s) [noise=true , address=89.149.201.147:4010 , piece_id=RY3V22SFDMDJL2SA67IVSIA3HL5ZXWFPDFEGKVTN3OJ6A3EY257Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 159.223.143.39:56278->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 19:04:28.508664395 (238h48m29.312023562s) [noise=true , address=89.149.201.147:4001 , piece_id=ORKT2SIHXBZWT626NROJIJGONLRAI3YOFHAFDQ7JOOEQMHNDPAPQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8, address:89.149.201.147:4001): protocol: read tcp 159.223.143.39:49320->89.149.201.147:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 19:04:28.508923513 (238h48m29.312282696s) [noise=true , address=89.149.201.147:4023 , piece_id=NWUSBCFDTVH67AO5RFAUMRBP7I4BA6LM2SS46REMDANS5PULE33A, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv, address:89.149.201.147:4023): protocol: read tcp 159.223.143.39:48854->89.149.201.147:4023: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Feb 27 19:04:28.508961365 (238h48m29.312320533s) [seg_pos={0 0}, upload=248562]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 19:04:28.611109748 (238h48m29.414468923s) [upload=248562]: Done uploading segments...
Feb 27 19:04:28.611159571 (238h48m29.414518748s) [upload=248562]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.040068208s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
28.040992ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Length=284 Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18983007517E56B6 X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:07:41 GMT Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=E6Zj8BKEo5Q Accept-Ranges=bytes target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/711cd5682b18a17c
FAIL (2026-02-27 19:07:41 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18983007517E56B6</RequestId><HostId></HostId></Error>"
Duration: 28.064797ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.209091ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Security-Policy=block-all-mixed-content Server=Storj X-Amz-Request-Id=1898300400786D05 Date=Fri, 27 Feb 2026 19:07:27 GMT Accept-Ranges=bytes Content-Type=application/xml Vary=Origin X-Request-Id=cySo9W7MXD X-Xss-Protection=1; mode=block Content-Length=274 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/3f3425b40fe37b47
FAIL (2026-02-27 19:07:27 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898300400786D05</RequestId><HostId></HostId></Error>"
Duration: 10.236671ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
14.649812ms remote:
"tcp:136.0.77.3:443" response headers:
Server=Storj Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Vary=Origin X-Amz-Request-Id=18983001F7FB1DC0 X-Request-Id=T9W4kCCzKo2 X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:07:18 GMT Content-Type=application/xml target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/673a6bc5e072f11d
FAIL (2026-02-27 19:07:18 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18983001F7FB1DC0</RequestId><HostId></HostId></Error>"
Duration: 14.675419ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..X...................................................X.....] duration:
2.0807575s trace:
https://jaeger.dev.storj.io/trace/5ba7b9ab33770c8
FAIL (2026-02-27 19:07:07 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
ess=185.167.204.205:4020 , piece_id=BRCPVT4JBAXGNYY7N547DSMTC7L2IKGG7RHT7MU7ISL2NC3ZDBCQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.881920320 (238h47m29.685279497s) [noise=true , address=185.167.204.206:4046 , piece_id=6ZTW47UKRKWVYGJHOBHANVWZWE6PDHRLITMD5VUNCARVASHK47AQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882277613 (238h47m29.685636797s) [noise=true , address=81.171.14.70:4025 , piece_id=JAKRF6PR6HKFBA536E3EQQ3WCWKDTYUXXCWG4THF5AFMACWK5H5Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882304171 (238h47m29.685663353s) [noise=true , address=89.149.201.147:4001 , piece_id=ORKT2SIHXBZWT626NROJIJGONLRAI3YOFHAFDQ7JOOEQMHNDPAPQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882721732 (238h47m29.68608091s) [noise=true , address=81.171.14.70:4038 , piece_id=KKVAZ7XUADG7JYHVSIXKTHJDXKCUICAGRZSBZHCSPD36LW6OYT2Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.883170726 (238h47m29.686529903s) [noise=true , address=89.149.201.147:4017 , piece_id=EQXPHI4WOCMNHULZT3KP4RXHHCUVM77FGPPLOQOSBODSKAE4NVRQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.883675356 (238h47m29.687034539s) [noise=true , address=185.167.204.206:4040 , piece_id=NBIGXDP7K2G2DYUTEHCSML4X5QF6DWITXNOX6G6LJCWOLUZZJWHA, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.884309144 (238h47m29.687668328s) [noise=true , address=89.149.201.147:4023 , piece_id=NWUSBCFDTVH67AO5RFAUMRBP7I4BA6LM2SS46REMDANS5PULE33A, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.884827042 (238h47m29.68818621s) [noise=true , address=185.167.204.206:4041 , piece_id=PNMLNBEI5MCZSXIELZF22U7C5FBTJ3NKZVSMHHBZ4NHRO4CE2JMQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.887870537 (238h47m29.691229718s) [noise=true , address=89.149.201.147:4010 , piece_id=RY3V22SFDMDJL2SA67IVSIA3HL5ZXWFPDFEGKVTN3OJ6A3EY257Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.888962425 (238h47m29.692321668s) [noise=true , address=81.171.14.70:4030 , piece_id=RXX3L2JLRR63LJMPPHD4KIB4S3ETKXPJGZ7SO7YIVEPNX7J4SOUA, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.889823462 (238h47m29.693182639s) [noise=true , address=185.167.204.205:4016 , piece_id=TKSQKNTBVIDDBR3UPJLX7ZCNMFMEXYPXL7YJCQ4ZSPYG2H76PY3A, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:29.305942479 (238h47m30.109301644s) [upload=248562]: Next returned nil segment
Feb 27 19:03:29.305967164 (238h47m30.109326339s) [upload=248562]: Waiting for error group managing segments...
Feb 27 19:03:29.485292528 (238h47m30.288651714s) [noise=true , address=81.171.14.70:4032 , piece_id=GVGNNAVZFDEQPI6BT6ZRDOWEDEJGTI5CRWZQDQ7ROPCYV4WU2OMQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.488867142 (238h47m30.292226317s) [noise=true , address=81.171.14.70:4042 , piece_id=JK6BODBCGQGSFSFPS4DADNUW4SLFV2ERJPWT6IZVVGD5JNBQXKQA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.488939564 (238h47m30.292298726s) [noise=true , address=81.171.14.70:4038 , piece_id=KKVAZ7XUADG7JYHVSIXKTHJDXKCUICAGRZSBZHCSPD36LW6OYT2Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.493270337 (238h47m30.29662951s) [noise=true , address=81.171.14.70:4025 , piece_id=JAKRF6PR6HKFBA536E3EQQ3WCWKDTYUXXCWG4THF5AFMACWK5H5Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.494182345 (238h47m30.297541509s) [noise=true , address=81.171.14.70:4030 , piece_id=RXX3L2JLRR63LJMPPHD4KIB4S3ETKXPJGZ7SO7YIVEPNX7J4SOUA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.495569378 (238h47m30.298928546s) [noise=true , address=81.171.14.70:4034 , piece_id=TKUZL52SXKKF4KUVCSTYIMJTTBZZ5NZ7WYQSLH4Z4DMYQD2GLWNA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.497221478 (238h47m30.300580654s) [noise=true , address=185.167.204.205:4016 , piece_id=TKSQKNTBVIDDBR3UPJLX7ZCNMFMEXYPXL7YJCQ4ZSPYG2H76PY3A, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.498481050 (238h47m30.301840223s) [noise=true , address=81.171.14.70:4047 , piece_id=XSDESMXZSX2FPVCIM6X533ZVBQ2NRCKCUSJ7UBEOCG76RCVOFFQA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.499128725 (238h47m30.302487899s) [noise=true , address=81.171.14.70:4033 , piece_id=UTG5PYLTCAKPYWHBFIEHYPQGWX7V2CWPPZUS3IHHM6IWHG2RSYIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.500770132 (238h47m30.304129311s) [noise=true , address=185.167.204.205:4011 , piece_id=BGJAKT35K6HYKSBM75R7Y527P2WEJHTWDZNVVYZOKLZOD3GWQDTA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.520282724 (238h47m30.323641898s) [noise=true , address=89.149.201.147:4024 , piece_id=SPWYXRSYXBFJVRYRX7KUN2PPIKMOJAM5PEJJMDTU7MCO52BBOGIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.520458144 (238h47m30.323817325s) [noise=true , address=89.149.201.147:4017 , piece_id=EQXPHI4WOCMNHULZT3KP4RXHHCUVM77FGPPLOQOSBODSKAE4NVRQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.528608534 (238h47m30.331967713s) [noise=true , address=185.167.204.206:4040 , piece_id=NBIGXDP7K2G2DYUTEHCSML4X5QF6DWITXNOX6G6LJCWOLUZZJWHA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.530517514 (238h47m30.333876679s) [noise=true , address=81.171.14.70:4029 , piece_id=P4PKJ6ZBWW5XJQ3ABVC5KDGVROK6TWZFEYOLJIMVO4GH4P27NY5Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.540229723 (238h47m30.343588896s) [noise=true , address=185.167.204.206:4039 , piece_id=B4BO7RY4U3QHZKNAAGUKFVLO3F7SCHPYM4TNUXBYM3UWIFC6RSIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.549193658 (238h47m30.352552843s) [noise=true , address=185.167.204.206:4037 , piece_id=4QRWKI66B2SVG2OWNQ5MQMQO33ZLWEV6OABDFMQWDNJVHWMRTJFQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.573203429 (238h47m30.376562603s) [noise=true , address=185.167.204.205:4007 , piece_id=JSFJFE6J7T3OYZUCU6OSGBTOGTDKXWKUX2GKZBZJPC4XMVUHI2RA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.578612555 (238h47m30.381971729s) [noise=true , address=185.167.204.206:4072 , piece_id=NWCN7VZ7C77J32GXW7MUFMACI3V45J7YQPRWIVNICO5E6N6Q5DBA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.578869362 (238h47m30.382228531s) [noise=true , address=185.167.204.206:4041 , piece_id=PNMLNBEI5MCZSXIELZF22U7C5FBTJ3NKZVSMHHBZ4NHRO4CE2JMQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.585453146 (238h47m30.388812327s) [noise=true , address=89.149.201.147:4004 , piece_id=R6KLLIMXVNYBDWH2W5KCOBWCQYF7RO47TFQKPHOR3UTYY446ATQQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.585510255 (238h47m30.388869432s) [noise=true , address=185.167.204.205:4009 , piece_id=UW66SRSI27LFSDXBPXZVA4SOIXQSEDIQAGCCPNFOIDYC6CO5DNXA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.607509471 (238h47m30.41086865s) [noise=true , address=185.167.204.206:4053 , piece_id=7CEFYGFPOKLIDAZWBEPBJ43O4OVYYUF66WABXWM7RADT4OBQ4AWQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.622982646 (238h47m30.426341819s) [noise=true , address=185.167.204.206:4046 , piece_id=6ZTW47UKRKWVYGJHOBHANVWZWE6PDHRLITMD5VUNCARVASHK47AQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.635158349 (238h47m30.438517522s) [noise=true , address=185.167.204.205:4020 , piece_id=BRCPVT4JBAXGNYY7N547DSMTC7L2IKGG7RHT7MU7ISL2NC3ZDBCQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:35.525495067 (238h47m36.328854242s) [noise=true , address=185.167.204.206:4044 , piece_id=2XMMX3NSARIQBL6SHBUTLTCTN7VC5HNHW5B5VGX4SBKNJTUDBSNA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:04:28.508369911 (238h48m29.311729091s) [noise=true , address=89.149.201.147:4010 , piece_id=RY3V22SFDMDJL2SA67IVSIA3HL5ZXWFPDFEGKVTN3OJ6A3EY257Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 159.223.143.39:56278->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 19:04:28.508664395 (238h48m29.312023562s) [noise=true , address=89.149.201.147:4001 , piece_id=ORKT2SIHXBZWT626NROJIJGONLRAI3YOFHAFDQ7JOOEQMHNDPAPQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8, address:89.149.201.147:4001): protocol: read tcp 159.223.143.39:49320->89.149.201.147:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 19:04:28.508923513 (238h48m29.312282696s) [noise=true , address=89.149.201.147:4023 , piece_id=NWUSBCFDTVH67AO5RFAUMRBP7I4BA6LM2SS46REMDANS5PULE33A, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv, address:89.149.201.147:4023): protocol: read tcp 159.223.143.39:48854->89.149.201.147:4023: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Feb 27 19:04:28.508961365 (238h48m29.312320533s) [seg_pos={0 0}, upload=248562]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 19:04:28.611109748 (238h48m29.414468923s) [upload=248562]: Done uploading segments...
Feb 27 19:04:28.611159571 (238h48m29.414518748s) [upload=248562]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.080805902s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
25.770899ms remote:
"tcp:136.0.77.3:443" response headers:
Server=Storj Vary=Origin X-Amz-Request-Id=18982FFD78F6E013 X-Request-Id=NoPRqbUFHcN Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:06:59 GMT Accept-Ranges=bytes target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/7d5118ad09506aa2
FAIL (2026-02-27 19:06:59 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FFD78F6E013</RequestId><HostId></HostId></Error>"
Duration: 25.802706ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
32.153586ms remote:
"tcp:45.38.24.2:443" response headers:
X-Request-Id=biQryowAM3N Content-Security-Policy=block-all-mixed-content X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:06:47 GMT Accept-Ranges=bytes Content-Length=284 Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=18982FFACCB8BB13 target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/55de320af673a5bd
FAIL (2026-02-27 19:06:47 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FFACCB8BB13</RequestId><HostId></HostId></Error>"
Duration: 32.181964ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
12.404749ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Length=274 Content-Security-Policy=block-all-mixed-content X-Request-Id=RuPfkJyF8xU Date=Fri, 27 Feb 2026 19:06:39 GMT Accept-Ranges=bytes Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=18982FF8DF3E6F2A X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/3f4a87b3601ff2a
FAIL (2026-02-27 19:06:39 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FF8DF3E6F2A</RequestId><HostId></HostId></Error>"
Duration: 12.423179ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.835256ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Length=274 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=18982FF7D1386D78 Accept-Ranges=bytes Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=Gmkr5PefEnc X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:06:34 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/61dd9c94fde16bec
FAIL (2026-02-27 19:06:34 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FF7D1386D78</RequestId><HostId></HostId></Error>"
Duration: 11.89061ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...X...................................................X....] duration:
2.052464696s trace:
https://jaeger.dev.storj.io/trace/5ba7b9ab33770c8
FAIL (2026-02-27 19:06:31 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
ess=185.167.204.205:4020 , piece_id=BRCPVT4JBAXGNYY7N547DSMTC7L2IKGG7RHT7MU7ISL2NC3ZDBCQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.881920320 (238h47m29.685279497s) [noise=true , address=185.167.204.206:4046 , piece_id=6ZTW47UKRKWVYGJHOBHANVWZWE6PDHRLITMD5VUNCARVASHK47AQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882277613 (238h47m29.685636797s) [noise=true , address=81.171.14.70:4025 , piece_id=JAKRF6PR6HKFBA536E3EQQ3WCWKDTYUXXCWG4THF5AFMACWK5H5Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882304171 (238h47m29.685663353s) [noise=true , address=89.149.201.147:4001 , piece_id=ORKT2SIHXBZWT626NROJIJGONLRAI3YOFHAFDQ7JOOEQMHNDPAPQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882721732 (238h47m29.68608091s) [noise=true , address=81.171.14.70:4038 , piece_id=KKVAZ7XUADG7JYHVSIXKTHJDXKCUICAGRZSBZHCSPD36LW6OYT2Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.883170726 (238h47m29.686529903s) [noise=true , address=89.149.201.147:4017 , piece_id=EQXPHI4WOCMNHULZT3KP4RXHHCUVM77FGPPLOQOSBODSKAE4NVRQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.883675356 (238h47m29.687034539s) [noise=true , address=185.167.204.206:4040 , piece_id=NBIGXDP7K2G2DYUTEHCSML4X5QF6DWITXNOX6G6LJCWOLUZZJWHA, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.884309144 (238h47m29.687668328s) [noise=true , address=89.149.201.147:4023 , piece_id=NWUSBCFDTVH67AO5RFAUMRBP7I4BA6LM2SS46REMDANS5PULE33A, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.884827042 (238h47m29.68818621s) [noise=true , address=185.167.204.206:4041 , piece_id=PNMLNBEI5MCZSXIELZF22U7C5FBTJ3NKZVSMHHBZ4NHRO4CE2JMQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.887870537 (238h47m29.691229718s) [noise=true , address=89.149.201.147:4010 , piece_id=RY3V22SFDMDJL2SA67IVSIA3HL5ZXWFPDFEGKVTN3OJ6A3EY257Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.888962425 (238h47m29.692321668s) [noise=true , address=81.171.14.70:4030 , piece_id=RXX3L2JLRR63LJMPPHD4KIB4S3ETKXPJGZ7SO7YIVEPNX7J4SOUA, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.889823462 (238h47m29.693182639s) [noise=true , address=185.167.204.205:4016 , piece_id=TKSQKNTBVIDDBR3UPJLX7ZCNMFMEXYPXL7YJCQ4ZSPYG2H76PY3A, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:29.305942479 (238h47m30.109301644s) [upload=248562]: Next returned nil segment
Feb 27 19:03:29.305967164 (238h47m30.109326339s) [upload=248562]: Waiting for error group managing segments...
Feb 27 19:03:29.485292528 (238h47m30.288651714s) [noise=true , address=81.171.14.70:4032 , piece_id=GVGNNAVZFDEQPI6BT6ZRDOWEDEJGTI5CRWZQDQ7ROPCYV4WU2OMQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.488867142 (238h47m30.292226317s) [noise=true , address=81.171.14.70:4042 , piece_id=JK6BODBCGQGSFSFPS4DADNUW4SLFV2ERJPWT6IZVVGD5JNBQXKQA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.488939564 (238h47m30.292298726s) [noise=true , address=81.171.14.70:4038 , piece_id=KKVAZ7XUADG7JYHVSIXKTHJDXKCUICAGRZSBZHCSPD36LW6OYT2Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.493270337 (238h47m30.29662951s) [noise=true , address=81.171.14.70:4025 , piece_id=JAKRF6PR6HKFBA536E3EQQ3WCWKDTYUXXCWG4THF5AFMACWK5H5Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.494182345 (238h47m30.297541509s) [noise=true , address=81.171.14.70:4030 , piece_id=RXX3L2JLRR63LJMPPHD4KIB4S3ETKXPJGZ7SO7YIVEPNX7J4SOUA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.495569378 (238h47m30.298928546s) [noise=true , address=81.171.14.70:4034 , piece_id=TKUZL52SXKKF4KUVCSTYIMJTTBZZ5NZ7WYQSLH4Z4DMYQD2GLWNA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.497221478 (238h47m30.300580654s) [noise=true , address=185.167.204.205:4016 , piece_id=TKSQKNTBVIDDBR3UPJLX7ZCNMFMEXYPXL7YJCQ4ZSPYG2H76PY3A, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.498481050 (238h47m30.301840223s) [noise=true , address=81.171.14.70:4047 , piece_id=XSDESMXZSX2FPVCIM6X533ZVBQ2NRCKCUSJ7UBEOCG76RCVOFFQA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.499128725 (238h47m30.302487899s) [noise=true , address=81.171.14.70:4033 , piece_id=UTG5PYLTCAKPYWHBFIEHYPQGWX7V2CWPPZUS3IHHM6IWHG2RSYIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.500770132 (238h47m30.304129311s) [noise=true , address=185.167.204.205:4011 , piece_id=BGJAKT35K6HYKSBM75R7Y527P2WEJHTWDZNVVYZOKLZOD3GWQDTA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.520282724 (238h47m30.323641898s) [noise=true , address=89.149.201.147:4024 , piece_id=SPWYXRSYXBFJVRYRX7KUN2PPIKMOJAM5PEJJMDTU7MCO52BBOGIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.520458144 (238h47m30.323817325s) [noise=true , address=89.149.201.147:4017 , piece_id=EQXPHI4WOCMNHULZT3KP4RXHHCUVM77FGPPLOQOSBODSKAE4NVRQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.528608534 (238h47m30.331967713s) [noise=true , address=185.167.204.206:4040 , piece_id=NBIGXDP7K2G2DYUTEHCSML4X5QF6DWITXNOX6G6LJCWOLUZZJWHA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.530517514 (238h47m30.333876679s) [noise=true , address=81.171.14.70:4029 , piece_id=P4PKJ6ZBWW5XJQ3ABVC5KDGVROK6TWZFEYOLJIMVO4GH4P27NY5Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.540229723 (238h47m30.343588896s) [noise=true , address=185.167.204.206:4039 , piece_id=B4BO7RY4U3QHZKNAAGUKFVLO3F7SCHPYM4TNUXBYM3UWIFC6RSIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.549193658 (238h47m30.352552843s) [noise=true , address=185.167.204.206:4037 , piece_id=4QRWKI66B2SVG2OWNQ5MQMQO33ZLWEV6OABDFMQWDNJVHWMRTJFQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.573203429 (238h47m30.376562603s) [noise=true , address=185.167.204.205:4007 , piece_id=JSFJFE6J7T3OYZUCU6OSGBTOGTDKXWKUX2GKZBZJPC4XMVUHI2RA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.578612555 (238h47m30.381971729s) [noise=true , address=185.167.204.206:4072 , piece_id=NWCN7VZ7C77J32GXW7MUFMACI3V45J7YQPRWIVNICO5E6N6Q5DBA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.578869362 (238h47m30.382228531s) [noise=true , address=185.167.204.206:4041 , piece_id=PNMLNBEI5MCZSXIELZF22U7C5FBTJ3NKZVSMHHBZ4NHRO4CE2JMQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.585453146 (238h47m30.388812327s) [noise=true , address=89.149.201.147:4004 , piece_id=R6KLLIMXVNYBDWH2W5KCOBWCQYF7RO47TFQKPHOR3UTYY446ATQQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.585510255 (238h47m30.388869432s) [noise=true , address=185.167.204.205:4009 , piece_id=UW66SRSI27LFSDXBPXZVA4SOIXQSEDIQAGCCPNFOIDYC6CO5DNXA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.607509471 (238h47m30.41086865s) [noise=true , address=185.167.204.206:4053 , piece_id=7CEFYGFPOKLIDAZWBEPBJ43O4OVYYUF66WABXWM7RADT4OBQ4AWQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.622982646 (238h47m30.426341819s) [noise=true , address=185.167.204.206:4046 , piece_id=6ZTW47UKRKWVYGJHOBHANVWZWE6PDHRLITMD5VUNCARVASHK47AQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.635158349 (238h47m30.438517522s) [noise=true , address=185.167.204.205:4020 , piece_id=BRCPVT4JBAXGNYY7N547DSMTC7L2IKGG7RHT7MU7ISL2NC3ZDBCQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:35.525495067 (238h47m36.328854242s) [noise=true , address=185.167.204.206:4044 , piece_id=2XMMX3NSARIQBL6SHBUTLTCTN7VC5HNHW5B5VGX4SBKNJTUDBSNA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:04:28.508369911 (238h48m29.311729091s) [noise=true , address=89.149.201.147:4010 , piece_id=RY3V22SFDMDJL2SA67IVSIA3HL5ZXWFPDFEGKVTN3OJ6A3EY257Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 159.223.143.39:56278->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 19:04:28.508664395 (238h48m29.312023562s) [noise=true , address=89.149.201.147:4001 , piece_id=ORKT2SIHXBZWT626NROJIJGONLRAI3YOFHAFDQ7JOOEQMHNDPAPQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8, address:89.149.201.147:4001): protocol: read tcp 159.223.143.39:49320->89.149.201.147:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 19:04:28.508923513 (238h48m29.312282696s) [noise=true , address=89.149.201.147:4023 , piece_id=NWUSBCFDTVH67AO5RFAUMRBP7I4BA6LM2SS46REMDANS5PULE33A, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv, address:89.149.201.147:4023): protocol: read tcp 159.223.143.39:48854->89.149.201.147:4023: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Feb 27 19:04:28.508961365 (238h48m29.312320533s) [seg_pos={0 0}, upload=248562]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 19:04:28.611109748 (238h48m29.414468923s) [upload=248562]: Done uploading segments...
Feb 27 19:04:28.611159571 (238h48m29.414518748s) [upload=248562]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.052492266s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
15.146963ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Length=274 Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=18982FF37271E352 X-Request-Id=USqFpbiKCZJ X-Xss-Protection=1; mode=block Content-Security-Policy=block-all-mixed-content Date=Fri, 27 Feb 2026 19:06:16 GMT Accept-Ranges=bytes target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/185fc9428164bf46
FAIL (2026-02-27 19:06:16 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FF37271E352</RequestId><HostId></HostId></Error>"
Duration: 15.17411ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [....X...................................................X...] duration:
2.061173132s trace:
https://jaeger.dev.storj.io/trace/5ba7b9ab33770c8
FAIL (2026-02-27 19:06:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
ess=185.167.204.205:4020 , piece_id=BRCPVT4JBAXGNYY7N547DSMTC7L2IKGG7RHT7MU7ISL2NC3ZDBCQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.881920320 (238h47m29.685279497s) [noise=true , address=185.167.204.206:4046 , piece_id=6ZTW47UKRKWVYGJHOBHANVWZWE6PDHRLITMD5VUNCARVASHK47AQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882277613 (238h47m29.685636797s) [noise=true , address=81.171.14.70:4025 , piece_id=JAKRF6PR6HKFBA536E3EQQ3WCWKDTYUXXCWG4THF5AFMACWK5H5Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882304171 (238h47m29.685663353s) [noise=true , address=89.149.201.147:4001 , piece_id=ORKT2SIHXBZWT626NROJIJGONLRAI3YOFHAFDQ7JOOEQMHNDPAPQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882721732 (238h47m29.68608091s) [noise=true , address=81.171.14.70:4038 , piece_id=KKVAZ7XUADG7JYHVSIXKTHJDXKCUICAGRZSBZHCSPD36LW6OYT2Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.883170726 (238h47m29.686529903s) [noise=true , address=89.149.201.147:4017 , piece_id=EQXPHI4WOCMNHULZT3KP4RXHHCUVM77FGPPLOQOSBODSKAE4NVRQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.883675356 (238h47m29.687034539s) [noise=true , address=185.167.204.206:4040 , piece_id=NBIGXDP7K2G2DYUTEHCSML4X5QF6DWITXNOX6G6LJCWOLUZZJWHA, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.884309144 (238h47m29.687668328s) [noise=true , address=89.149.201.147:4023 , piece_id=NWUSBCFDTVH67AO5RFAUMRBP7I4BA6LM2SS46REMDANS5PULE33A, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.884827042 (238h47m29.68818621s) [noise=true , address=185.167.204.206:4041 , piece_id=PNMLNBEI5MCZSXIELZF22U7C5FBTJ3NKZVSMHHBZ4NHRO4CE2JMQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.887870537 (238h47m29.691229718s) [noise=true , address=89.149.201.147:4010 , piece_id=RY3V22SFDMDJL2SA67IVSIA3HL5ZXWFPDFEGKVTN3OJ6A3EY257Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.888962425 (238h47m29.692321668s) [noise=true , address=81.171.14.70:4030 , piece_id=RXX3L2JLRR63LJMPPHD4KIB4S3ETKXPJGZ7SO7YIVEPNX7J4SOUA, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.889823462 (238h47m29.693182639s) [noise=true , address=185.167.204.205:4016 , piece_id=TKSQKNTBVIDDBR3UPJLX7ZCNMFMEXYPXL7YJCQ4ZSPYG2H76PY3A, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:29.305942479 (238h47m30.109301644s) [upload=248562]: Next returned nil segment
Feb 27 19:03:29.305967164 (238h47m30.109326339s) [upload=248562]: Waiting for error group managing segments...
Feb 27 19:03:29.485292528 (238h47m30.288651714s) [noise=true , address=81.171.14.70:4032 , piece_id=GVGNNAVZFDEQPI6BT6ZRDOWEDEJGTI5CRWZQDQ7ROPCYV4WU2OMQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.488867142 (238h47m30.292226317s) [noise=true , address=81.171.14.70:4042 , piece_id=JK6BODBCGQGSFSFPS4DADNUW4SLFV2ERJPWT6IZVVGD5JNBQXKQA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.488939564 (238h47m30.292298726s) [noise=true , address=81.171.14.70:4038 , piece_id=KKVAZ7XUADG7JYHVSIXKTHJDXKCUICAGRZSBZHCSPD36LW6OYT2Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.493270337 (238h47m30.29662951s) [noise=true , address=81.171.14.70:4025 , piece_id=JAKRF6PR6HKFBA536E3EQQ3WCWKDTYUXXCWG4THF5AFMACWK5H5Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.494182345 (238h47m30.297541509s) [noise=true , address=81.171.14.70:4030 , piece_id=RXX3L2JLRR63LJMPPHD4KIB4S3ETKXPJGZ7SO7YIVEPNX7J4SOUA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.495569378 (238h47m30.298928546s) [noise=true , address=81.171.14.70:4034 , piece_id=TKUZL52SXKKF4KUVCSTYIMJTTBZZ5NZ7WYQSLH4Z4DMYQD2GLWNA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.497221478 (238h47m30.300580654s) [noise=true , address=185.167.204.205:4016 , piece_id=TKSQKNTBVIDDBR3UPJLX7ZCNMFMEXYPXL7YJCQ4ZSPYG2H76PY3A, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.498481050 (238h47m30.301840223s) [noise=true , address=81.171.14.70:4047 , piece_id=XSDESMXZSX2FPVCIM6X533ZVBQ2NRCKCUSJ7UBEOCG76RCVOFFQA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.499128725 (238h47m30.302487899s) [noise=true , address=81.171.14.70:4033 , piece_id=UTG5PYLTCAKPYWHBFIEHYPQGWX7V2CWPPZUS3IHHM6IWHG2RSYIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.500770132 (238h47m30.304129311s) [noise=true , address=185.167.204.205:4011 , piece_id=BGJAKT35K6HYKSBM75R7Y527P2WEJHTWDZNVVYZOKLZOD3GWQDTA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.520282724 (238h47m30.323641898s) [noise=true , address=89.149.201.147:4024 , piece_id=SPWYXRSYXBFJVRYRX7KUN2PPIKMOJAM5PEJJMDTU7MCO52BBOGIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.520458144 (238h47m30.323817325s) [noise=true , address=89.149.201.147:4017 , piece_id=EQXPHI4WOCMNHULZT3KP4RXHHCUVM77FGPPLOQOSBODSKAE4NVRQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.528608534 (238h47m30.331967713s) [noise=true , address=185.167.204.206:4040 , piece_id=NBIGXDP7K2G2DYUTEHCSML4X5QF6DWITXNOX6G6LJCWOLUZZJWHA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.530517514 (238h47m30.333876679s) [noise=true , address=81.171.14.70:4029 , piece_id=P4PKJ6ZBWW5XJQ3ABVC5KDGVROK6TWZFEYOLJIMVO4GH4P27NY5Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.540229723 (238h47m30.343588896s) [noise=true , address=185.167.204.206:4039 , piece_id=B4BO7RY4U3QHZKNAAGUKFVLO3F7SCHPYM4TNUXBYM3UWIFC6RSIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.549193658 (238h47m30.352552843s) [noise=true , address=185.167.204.206:4037 , piece_id=4QRWKI66B2SVG2OWNQ5MQMQO33ZLWEV6OABDFMQWDNJVHWMRTJFQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.573203429 (238h47m30.376562603s) [noise=true , address=185.167.204.205:4007 , piece_id=JSFJFE6J7T3OYZUCU6OSGBTOGTDKXWKUX2GKZBZJPC4XMVUHI2RA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.578612555 (238h47m30.381971729s) [noise=true , address=185.167.204.206:4072 , piece_id=NWCN7VZ7C77J32GXW7MUFMACI3V45J7YQPRWIVNICO5E6N6Q5DBA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.578869362 (238h47m30.382228531s) [noise=true , address=185.167.204.206:4041 , piece_id=PNMLNBEI5MCZSXIELZF22U7C5FBTJ3NKZVSMHHBZ4NHRO4CE2JMQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.585453146 (238h47m30.388812327s) [noise=true , address=89.149.201.147:4004 , piece_id=R6KLLIMXVNYBDWH2W5KCOBWCQYF7RO47TFQKPHOR3UTYY446ATQQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.585510255 (238h47m30.388869432s) [noise=true , address=185.167.204.205:4009 , piece_id=UW66SRSI27LFSDXBPXZVA4SOIXQSEDIQAGCCPNFOIDYC6CO5DNXA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.607509471 (238h47m30.41086865s) [noise=true , address=185.167.204.206:4053 , piece_id=7CEFYGFPOKLIDAZWBEPBJ43O4OVYYUF66WABXWM7RADT4OBQ4AWQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.622982646 (238h47m30.426341819s) [noise=true , address=185.167.204.206:4046 , piece_id=6ZTW47UKRKWVYGJHOBHANVWZWE6PDHRLITMD5VUNCARVASHK47AQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.635158349 (238h47m30.438517522s) [noise=true , address=185.167.204.205:4020 , piece_id=BRCPVT4JBAXGNYY7N547DSMTC7L2IKGG7RHT7MU7ISL2NC3ZDBCQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:35.525495067 (238h47m36.328854242s) [noise=true , address=185.167.204.206:4044 , piece_id=2XMMX3NSARIQBL6SHBUTLTCTN7VC5HNHW5B5VGX4SBKNJTUDBSNA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:04:28.508369911 (238h48m29.311729091s) [noise=true , address=89.149.201.147:4010 , piece_id=RY3V22SFDMDJL2SA67IVSIA3HL5ZXWFPDFEGKVTN3OJ6A3EY257Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 159.223.143.39:56278->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 19:04:28.508664395 (238h48m29.312023562s) [noise=true , address=89.149.201.147:4001 , piece_id=ORKT2SIHXBZWT626NROJIJGONLRAI3YOFHAFDQ7JOOEQMHNDPAPQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8, address:89.149.201.147:4001): protocol: read tcp 159.223.143.39:49320->89.149.201.147:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 19:04:28.508923513 (238h48m29.312282696s) [noise=true , address=89.149.201.147:4023 , piece_id=NWUSBCFDTVH67AO5RFAUMRBP7I4BA6LM2SS46REMDANS5PULE33A, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv, address:89.149.201.147:4023): protocol: read tcp 159.223.143.39:48854->89.149.201.147:4023: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Feb 27 19:04:28.508961365 (238h48m29.312320533s) [seg_pos={0 0}, upload=248562]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 19:04:28.611109748 (238h48m29.414468923s) [upload=248562]: Done uploading segments...
Feb 27 19:04:28.611159571 (238h48m29.414518748s) [upload=248562]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.061200927s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.385283ms remote:
"tcp:136.0.77.3:443" response headers:
X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:05:57 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18982FEF302BBDDA Content-Length=274 Server=Storj X-Request-Id=2GkCwxdDBZF target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/2e286a8e2ea18b4f
FAIL (2026-02-27 19:05:57 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FEF302BBDDA</RequestId><HostId></HostId></Error>"
Duration: 9.417805ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.996647ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Type=application/xml Server=Storj Content-Length=274 Content-Security-Policy=block-all-mixed-content Vary=Origin X-Amz-Request-Id=18982FED75D1D35D X-Request-Id=4ZHKNFKvzfv X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:05:50 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/37f508be696292a2
FAIL (2026-02-27 19:05:50 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FED75D1D35D</RequestId><HostId></HostId></Error>"
Duration: 10.022917ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.226378ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Length=284 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Request-Id=9kD9Rx2Vr3p Accept-Ranges=bytes Server=Storj Vary=Origin X-Amz-Request-Id=18982FECCBBC79F3 X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:05:47 GMT target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/46c7b2bcd185c7a6
FAIL (2026-02-27 19:05:47 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FECCBBC79F3</RequestId><HostId></HostId></Error>"
Duration: 30.257773ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
12.175243ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml X-Amz-Request-Id=18982FE94F949D19 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Request-Id=7QcQqWDtweS X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:05:32 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/7787baa761aae4e4
FAIL (2026-02-27 19:05:32 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FE94F949D19</RequestId><HostId></HostId></Error>"
Duration: 12.218985ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
12.966741ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Security-Policy=block-all-mixed-content Vary=Origin X-Amz-Request-Id=18982FE872EB8CF6 X-Request-Id=BkEnGrACkfp X-Xss-Protection=1; mode=block Content-Type=application/xml Server=Storj Date=Fri, 27 Feb 2026 19:05:28 GMT Accept-Ranges=bytes Content-Length=274 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/1ab1a2e514f6d80c
FAIL (2026-02-27 19:05:28 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FE872EB8CF6</RequestId><HostId></HostId></Error>"
Duration: 12.992139ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.....X...................................................X..] duration:
2.128747943s trace:
https://jaeger.dev.storj.io/trace/5ba7b9ab33770c8
FAIL (2026-02-27 19:05:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
ess=185.167.204.205:4020 , piece_id=BRCPVT4JBAXGNYY7N547DSMTC7L2IKGG7RHT7MU7ISL2NC3ZDBCQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.881920320 (238h47m29.685279497s) [noise=true , address=185.167.204.206:4046 , piece_id=6ZTW47UKRKWVYGJHOBHANVWZWE6PDHRLITMD5VUNCARVASHK47AQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882277613 (238h47m29.685636797s) [noise=true , address=81.171.14.70:4025 , piece_id=JAKRF6PR6HKFBA536E3EQQ3WCWKDTYUXXCWG4THF5AFMACWK5H5Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882304171 (238h47m29.685663353s) [noise=true , address=89.149.201.147:4001 , piece_id=ORKT2SIHXBZWT626NROJIJGONLRAI3YOFHAFDQ7JOOEQMHNDPAPQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882721732 (238h47m29.68608091s) [noise=true , address=81.171.14.70:4038 , piece_id=KKVAZ7XUADG7JYHVSIXKTHJDXKCUICAGRZSBZHCSPD36LW6OYT2Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.883170726 (238h47m29.686529903s) [noise=true , address=89.149.201.147:4017 , piece_id=EQXPHI4WOCMNHULZT3KP4RXHHCUVM77FGPPLOQOSBODSKAE4NVRQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.883675356 (238h47m29.687034539s) [noise=true , address=185.167.204.206:4040 , piece_id=NBIGXDP7K2G2DYUTEHCSML4X5QF6DWITXNOX6G6LJCWOLUZZJWHA, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.884309144 (238h47m29.687668328s) [noise=true , address=89.149.201.147:4023 , piece_id=NWUSBCFDTVH67AO5RFAUMRBP7I4BA6LM2SS46REMDANS5PULE33A, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.884827042 (238h47m29.68818621s) [noise=true , address=185.167.204.206:4041 , piece_id=PNMLNBEI5MCZSXIELZF22U7C5FBTJ3NKZVSMHHBZ4NHRO4CE2JMQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.887870537 (238h47m29.691229718s) [noise=true , address=89.149.201.147:4010 , piece_id=RY3V22SFDMDJL2SA67IVSIA3HL5ZXWFPDFEGKVTN3OJ6A3EY257Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.888962425 (238h47m29.692321668s) [noise=true , address=81.171.14.70:4030 , piece_id=RXX3L2JLRR63LJMPPHD4KIB4S3ETKXPJGZ7SO7YIVEPNX7J4SOUA, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.889823462 (238h47m29.693182639s) [noise=true , address=185.167.204.205:4016 , piece_id=TKSQKNTBVIDDBR3UPJLX7ZCNMFMEXYPXL7YJCQ4ZSPYG2H76PY3A, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:29.305942479 (238h47m30.109301644s) [upload=248562]: Next returned nil segment
Feb 27 19:03:29.305967164 (238h47m30.109326339s) [upload=248562]: Waiting for error group managing segments...
Feb 27 19:03:29.485292528 (238h47m30.288651714s) [noise=true , address=81.171.14.70:4032 , piece_id=GVGNNAVZFDEQPI6BT6ZRDOWEDEJGTI5CRWZQDQ7ROPCYV4WU2OMQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.488867142 (238h47m30.292226317s) [noise=true , address=81.171.14.70:4042 , piece_id=JK6BODBCGQGSFSFPS4DADNUW4SLFV2ERJPWT6IZVVGD5JNBQXKQA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.488939564 (238h47m30.292298726s) [noise=true , address=81.171.14.70:4038 , piece_id=KKVAZ7XUADG7JYHVSIXKTHJDXKCUICAGRZSBZHCSPD36LW6OYT2Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.493270337 (238h47m30.29662951s) [noise=true , address=81.171.14.70:4025 , piece_id=JAKRF6PR6HKFBA536E3EQQ3WCWKDTYUXXCWG4THF5AFMACWK5H5Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.494182345 (238h47m30.297541509s) [noise=true , address=81.171.14.70:4030 , piece_id=RXX3L2JLRR63LJMPPHD4KIB4S3ETKXPJGZ7SO7YIVEPNX7J4SOUA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.495569378 (238h47m30.298928546s) [noise=true , address=81.171.14.70:4034 , piece_id=TKUZL52SXKKF4KUVCSTYIMJTTBZZ5NZ7WYQSLH4Z4DMYQD2GLWNA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.497221478 (238h47m30.300580654s) [noise=true , address=185.167.204.205:4016 , piece_id=TKSQKNTBVIDDBR3UPJLX7ZCNMFMEXYPXL7YJCQ4ZSPYG2H76PY3A, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.498481050 (238h47m30.301840223s) [noise=true , address=81.171.14.70:4047 , piece_id=XSDESMXZSX2FPVCIM6X533ZVBQ2NRCKCUSJ7UBEOCG76RCVOFFQA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.499128725 (238h47m30.302487899s) [noise=true , address=81.171.14.70:4033 , piece_id=UTG5PYLTCAKPYWHBFIEHYPQGWX7V2CWPPZUS3IHHM6IWHG2RSYIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.500770132 (238h47m30.304129311s) [noise=true , address=185.167.204.205:4011 , piece_id=BGJAKT35K6HYKSBM75R7Y527P2WEJHTWDZNVVYZOKLZOD3GWQDTA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.520282724 (238h47m30.323641898s) [noise=true , address=89.149.201.147:4024 , piece_id=SPWYXRSYXBFJVRYRX7KUN2PPIKMOJAM5PEJJMDTU7MCO52BBOGIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.520458144 (238h47m30.323817325s) [noise=true , address=89.149.201.147:4017 , piece_id=EQXPHI4WOCMNHULZT3KP4RXHHCUVM77FGPPLOQOSBODSKAE4NVRQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.528608534 (238h47m30.331967713s) [noise=true , address=185.167.204.206:4040 , piece_id=NBIGXDP7K2G2DYUTEHCSML4X5QF6DWITXNOX6G6LJCWOLUZZJWHA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.530517514 (238h47m30.333876679s) [noise=true , address=81.171.14.70:4029 , piece_id=P4PKJ6ZBWW5XJQ3ABVC5KDGVROK6TWZFEYOLJIMVO4GH4P27NY5Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.540229723 (238h47m30.343588896s) [noise=true , address=185.167.204.206:4039 , piece_id=B4BO7RY4U3QHZKNAAGUKFVLO3F7SCHPYM4TNUXBYM3UWIFC6RSIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.549193658 (238h47m30.352552843s) [noise=true , address=185.167.204.206:4037 , piece_id=4QRWKI66B2SVG2OWNQ5MQMQO33ZLWEV6OABDFMQWDNJVHWMRTJFQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.573203429 (238h47m30.376562603s) [noise=true , address=185.167.204.205:4007 , piece_id=JSFJFE6J7T3OYZUCU6OSGBTOGTDKXWKUX2GKZBZJPC4XMVUHI2RA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.578612555 (238h47m30.381971729s) [noise=true , address=185.167.204.206:4072 , piece_id=NWCN7VZ7C77J32GXW7MUFMACI3V45J7YQPRWIVNICO5E6N6Q5DBA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.578869362 (238h47m30.382228531s) [noise=true , address=185.167.204.206:4041 , piece_id=PNMLNBEI5MCZSXIELZF22U7C5FBTJ3NKZVSMHHBZ4NHRO4CE2JMQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.585453146 (238h47m30.388812327s) [noise=true , address=89.149.201.147:4004 , piece_id=R6KLLIMXVNYBDWH2W5KCOBWCQYF7RO47TFQKPHOR3UTYY446ATQQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.585510255 (238h47m30.388869432s) [noise=true , address=185.167.204.205:4009 , piece_id=UW66SRSI27LFSDXBPXZVA4SOIXQSEDIQAGCCPNFOIDYC6CO5DNXA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.607509471 (238h47m30.41086865s) [noise=true , address=185.167.204.206:4053 , piece_id=7CEFYGFPOKLIDAZWBEPBJ43O4OVYYUF66WABXWM7RADT4OBQ4AWQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.622982646 (238h47m30.426341819s) [noise=true , address=185.167.204.206:4046 , piece_id=6ZTW47UKRKWVYGJHOBHANVWZWE6PDHRLITMD5VUNCARVASHK47AQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.635158349 (238h47m30.438517522s) [noise=true , address=185.167.204.205:4020 , piece_id=BRCPVT4JBAXGNYY7N547DSMTC7L2IKGG7RHT7MU7ISL2NC3ZDBCQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:35.525495067 (238h47m36.328854242s) [noise=true , address=185.167.204.206:4044 , piece_id=2XMMX3NSARIQBL6SHBUTLTCTN7VC5HNHW5B5VGX4SBKNJTUDBSNA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:04:28.508369911 (238h48m29.311729091s) [noise=true , address=89.149.201.147:4010 , piece_id=RY3V22SFDMDJL2SA67IVSIA3HL5ZXWFPDFEGKVTN3OJ6A3EY257Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 159.223.143.39:56278->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 19:04:28.508664395 (238h48m29.312023562s) [noise=true , address=89.149.201.147:4001 , piece_id=ORKT2SIHXBZWT626NROJIJGONLRAI3YOFHAFDQ7JOOEQMHNDPAPQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8, address:89.149.201.147:4001): protocol: read tcp 159.223.143.39:49320->89.149.201.147:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 19:04:28.508923513 (238h48m29.312282696s) [noise=true , address=89.149.201.147:4023 , piece_id=NWUSBCFDTVH67AO5RFAUMRBP7I4BA6LM2SS46REMDANS5PULE33A, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv, address:89.149.201.147:4023): protocol: read tcp 159.223.143.39:48854->89.149.201.147:4023: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Feb 27 19:04:28.508961365 (238h48m29.312320533s) [seg_pos={0 0}, upload=248562]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 19:04:28.611109748 (238h48m29.414468923s) [upload=248562]: Done uploading segments...
Feb 27 19:04:28.611159571 (238h48m29.414518748s) [upload=248562]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.128773018s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [......X...................................................X.] duration:
53.630641704s trace:
https://jaeger.dev.storj.io/trace/5ba7b9ab33770c8
FAIL (2026-02-27 19:05:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
ess=185.167.204.205:4020 , piece_id=BRCPVT4JBAXGNYY7N547DSMTC7L2IKGG7RHT7MU7ISL2NC3ZDBCQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.881920320 (238h47m29.685279497s) [noise=true , address=185.167.204.206:4046 , piece_id=6ZTW47UKRKWVYGJHOBHANVWZWE6PDHRLITMD5VUNCARVASHK47AQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882277613 (238h47m29.685636797s) [noise=true , address=81.171.14.70:4025 , piece_id=JAKRF6PR6HKFBA536E3EQQ3WCWKDTYUXXCWG4THF5AFMACWK5H5Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882304171 (238h47m29.685663353s) [noise=true , address=89.149.201.147:4001 , piece_id=ORKT2SIHXBZWT626NROJIJGONLRAI3YOFHAFDQ7JOOEQMHNDPAPQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882721732 (238h47m29.68608091s) [noise=true , address=81.171.14.70:4038 , piece_id=KKVAZ7XUADG7JYHVSIXKTHJDXKCUICAGRZSBZHCSPD36LW6OYT2Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.883170726 (238h47m29.686529903s) [noise=true , address=89.149.201.147:4017 , piece_id=EQXPHI4WOCMNHULZT3KP4RXHHCUVM77FGPPLOQOSBODSKAE4NVRQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.883675356 (238h47m29.687034539s) [noise=true , address=185.167.204.206:4040 , piece_id=NBIGXDP7K2G2DYUTEHCSML4X5QF6DWITXNOX6G6LJCWOLUZZJWHA, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.884309144 (238h47m29.687668328s) [noise=true , address=89.149.201.147:4023 , piece_id=NWUSBCFDTVH67AO5RFAUMRBP7I4BA6LM2SS46REMDANS5PULE33A, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.884827042 (238h47m29.68818621s) [noise=true , address=185.167.204.206:4041 , piece_id=PNMLNBEI5MCZSXIELZF22U7C5FBTJ3NKZVSMHHBZ4NHRO4CE2JMQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.887870537 (238h47m29.691229718s) [noise=true , address=89.149.201.147:4010 , piece_id=RY3V22SFDMDJL2SA67IVSIA3HL5ZXWFPDFEGKVTN3OJ6A3EY257Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.888962425 (238h47m29.692321668s) [noise=true , address=81.171.14.70:4030 , piece_id=RXX3L2JLRR63LJMPPHD4KIB4S3ETKXPJGZ7SO7YIVEPNX7J4SOUA, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.889823462 (238h47m29.693182639s) [noise=true , address=185.167.204.205:4016 , piece_id=TKSQKNTBVIDDBR3UPJLX7ZCNMFMEXYPXL7YJCQ4ZSPYG2H76PY3A, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:29.305942479 (238h47m30.109301644s) [upload=248562]: Next returned nil segment
Feb 27 19:03:29.305967164 (238h47m30.109326339s) [upload=248562]: Waiting for error group managing segments...
Feb 27 19:03:29.485292528 (238h47m30.288651714s) [noise=true , address=81.171.14.70:4032 , piece_id=GVGNNAVZFDEQPI6BT6ZRDOWEDEJGTI5CRWZQDQ7ROPCYV4WU2OMQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.488867142 (238h47m30.292226317s) [noise=true , address=81.171.14.70:4042 , piece_id=JK6BODBCGQGSFSFPS4DADNUW4SLFV2ERJPWT6IZVVGD5JNBQXKQA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.488939564 (238h47m30.292298726s) [noise=true , address=81.171.14.70:4038 , piece_id=KKVAZ7XUADG7JYHVSIXKTHJDXKCUICAGRZSBZHCSPD36LW6OYT2Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.493270337 (238h47m30.29662951s) [noise=true , address=81.171.14.70:4025 , piece_id=JAKRF6PR6HKFBA536E3EQQ3WCWKDTYUXXCWG4THF5AFMACWK5H5Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.494182345 (238h47m30.297541509s) [noise=true , address=81.171.14.70:4030 , piece_id=RXX3L2JLRR63LJMPPHD4KIB4S3ETKXPJGZ7SO7YIVEPNX7J4SOUA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.495569378 (238h47m30.298928546s) [noise=true , address=81.171.14.70:4034 , piece_id=TKUZL52SXKKF4KUVCSTYIMJTTBZZ5NZ7WYQSLH4Z4DMYQD2GLWNA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.497221478 (238h47m30.300580654s) [noise=true , address=185.167.204.205:4016 , piece_id=TKSQKNTBVIDDBR3UPJLX7ZCNMFMEXYPXL7YJCQ4ZSPYG2H76PY3A, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.498481050 (238h47m30.301840223s) [noise=true , address=81.171.14.70:4047 , piece_id=XSDESMXZSX2FPVCIM6X533ZVBQ2NRCKCUSJ7UBEOCG76RCVOFFQA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.499128725 (238h47m30.302487899s) [noise=true , address=81.171.14.70:4033 , piece_id=UTG5PYLTCAKPYWHBFIEHYPQGWX7V2CWPPZUS3IHHM6IWHG2RSYIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.500770132 (238h47m30.304129311s) [noise=true , address=185.167.204.205:4011 , piece_id=BGJAKT35K6HYKSBM75R7Y527P2WEJHTWDZNVVYZOKLZOD3GWQDTA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.520282724 (238h47m30.323641898s) [noise=true , address=89.149.201.147:4024 , piece_id=SPWYXRSYXBFJVRYRX7KUN2PPIKMOJAM5PEJJMDTU7MCO52BBOGIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.520458144 (238h47m30.323817325s) [noise=true , address=89.149.201.147:4017 , piece_id=EQXPHI4WOCMNHULZT3KP4RXHHCUVM77FGPPLOQOSBODSKAE4NVRQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.528608534 (238h47m30.331967713s) [noise=true , address=185.167.204.206:4040 , piece_id=NBIGXDP7K2G2DYUTEHCSML4X5QF6DWITXNOX6G6LJCWOLUZZJWHA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.530517514 (238h47m30.333876679s) [noise=true , address=81.171.14.70:4029 , piece_id=P4PKJ6ZBWW5XJQ3ABVC5KDGVROK6TWZFEYOLJIMVO4GH4P27NY5Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.540229723 (238h47m30.343588896s) [noise=true , address=185.167.204.206:4039 , piece_id=B4BO7RY4U3QHZKNAAGUKFVLO3F7SCHPYM4TNUXBYM3UWIFC6RSIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.549193658 (238h47m30.352552843s) [noise=true , address=185.167.204.206:4037 , piece_id=4QRWKI66B2SVG2OWNQ5MQMQO33ZLWEV6OABDFMQWDNJVHWMRTJFQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.573203429 (238h47m30.376562603s) [noise=true , address=185.167.204.205:4007 , piece_id=JSFJFE6J7T3OYZUCU6OSGBTOGTDKXWKUX2GKZBZJPC4XMVUHI2RA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.578612555 (238h47m30.381971729s) [noise=true , address=185.167.204.206:4072 , piece_id=NWCN7VZ7C77J32GXW7MUFMACI3V45J7YQPRWIVNICO5E6N6Q5DBA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.578869362 (238h47m30.382228531s) [noise=true , address=185.167.204.206:4041 , piece_id=PNMLNBEI5MCZSXIELZF22U7C5FBTJ3NKZVSMHHBZ4NHRO4CE2JMQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.585453146 (238h47m30.388812327s) [noise=true , address=89.149.201.147:4004 , piece_id=R6KLLIMXVNYBDWH2W5KCOBWCQYF7RO47TFQKPHOR3UTYY446ATQQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.585510255 (238h47m30.388869432s) [noise=true , address=185.167.204.205:4009 , piece_id=UW66SRSI27LFSDXBPXZVA4SOIXQSEDIQAGCCPNFOIDYC6CO5DNXA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.607509471 (238h47m30.41086865s) [noise=true , address=185.167.204.206:4053 , piece_id=7CEFYGFPOKLIDAZWBEPBJ43O4OVYYUF66WABXWM7RADT4OBQ4AWQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.622982646 (238h47m30.426341819s) [noise=true , address=185.167.204.206:4046 , piece_id=6ZTW47UKRKWVYGJHOBHANVWZWE6PDHRLITMD5VUNCARVASHK47AQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.635158349 (238h47m30.438517522s) [noise=true , address=185.167.204.205:4020 , piece_id=BRCPVT4JBAXGNYY7N547DSMTC7L2IKGG7RHT7MU7ISL2NC3ZDBCQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:35.525495067 (238h47m36.328854242s) [noise=true , address=185.167.204.206:4044 , piece_id=2XMMX3NSARIQBL6SHBUTLTCTN7VC5HNHW5B5VGX4SBKNJTUDBSNA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:04:28.508369911 (238h48m29.311729091s) [noise=true , address=89.149.201.147:4010 , piece_id=RY3V22SFDMDJL2SA67IVSIA3HL5ZXWFPDFEGKVTN3OJ6A3EY257Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 159.223.143.39:56278->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 19:04:28.508664395 (238h48m29.312023562s) [noise=true , address=89.149.201.147:4001 , piece_id=ORKT2SIHXBZWT626NROJIJGONLRAI3YOFHAFDQ7JOOEQMHNDPAPQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8, address:89.149.201.147:4001): protocol: read tcp 159.223.143.39:49320->89.149.201.147:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 19:04:28.508923513 (238h48m29.312282696s) [noise=true , address=89.149.201.147:4023 , piece_id=NWUSBCFDTVH67AO5RFAUMRBP7I4BA6LM2SS46REMDANS5PULE33A, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv, address:89.149.201.147:4023): protocol: read tcp 159.223.143.39:48854->89.149.201.147:4023: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Feb 27 19:04:28.508961365 (238h48m29.312320533s) [seg_pos={0 0}, upload=248562]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 19:04:28.611109748 (238h48m29.414468923s) [upload=248562]: Done uploading segments...
Feb 27 19:04:28.611159571 (238h48m29.414518748s) [upload=248562]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 53.630665092s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.088705ms remote:
"tcp:136.0.77.3:443" response headers:
Date=Fri, 27 Feb 2026 19:05:06 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Request-Id=b45V91BDusJ X-Xss-Protection=1; mode=block Content-Length=274 Vary=Origin X-Amz-Request-Id=18982FE33FE7F249 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/73773cd665a2351
FAIL (2026-02-27 19:05:06 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FE33FE7F249</RequestId><HostId></HostId></Error>"
Duration: 10.119959ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.356093ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:04:58 GMT Accept-Ranges=bytes Server=Storj X-Amz-Request-Id=18982FE15CD2ADFF X-Request-Id=6fmJstnq8Ab Content-Length=284 target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/286c20335d520499
FAIL (2026-02-27 19:04:58 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FE15CD2ADFF</RequestId><HostId></HostId></Error>"
Duration: 30.380899ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
13.229193ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18982FDD8435DD10 X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:04:41 GMT Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=X7qCTCbu5xg target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/484b5d4b248f3c
FAIL (2026-02-27 19:04:41 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FDD8435DD10</RequestId><HostId></HostId></Error>"
Duration: 13.274382ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.......X...................................................X] duration:
1m0.105408919s trace:
https://jaeger.dev.storj.io/trace/5ba7b9ab33770c8
FAIL (2026-02-27 19:04:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
ess=185.167.204.205:4020 , piece_id=BRCPVT4JBAXGNYY7N547DSMTC7L2IKGG7RHT7MU7ISL2NC3ZDBCQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.881920320 (238h47m29.685279497s) [noise=true , address=185.167.204.206:4046 , piece_id=6ZTW47UKRKWVYGJHOBHANVWZWE6PDHRLITMD5VUNCARVASHK47AQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882277613 (238h47m29.685636797s) [noise=true , address=81.171.14.70:4025 , piece_id=JAKRF6PR6HKFBA536E3EQQ3WCWKDTYUXXCWG4THF5AFMACWK5H5Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882304171 (238h47m29.685663353s) [noise=true , address=89.149.201.147:4001 , piece_id=ORKT2SIHXBZWT626NROJIJGONLRAI3YOFHAFDQ7JOOEQMHNDPAPQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.882721732 (238h47m29.68608091s) [noise=true , address=81.171.14.70:4038 , piece_id=KKVAZ7XUADG7JYHVSIXKTHJDXKCUICAGRZSBZHCSPD36LW6OYT2Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.883170726 (238h47m29.686529903s) [noise=true , address=89.149.201.147:4017 , piece_id=EQXPHI4WOCMNHULZT3KP4RXHHCUVM77FGPPLOQOSBODSKAE4NVRQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.883675356 (238h47m29.687034539s) [noise=true , address=185.167.204.206:4040 , piece_id=NBIGXDP7K2G2DYUTEHCSML4X5QF6DWITXNOX6G6LJCWOLUZZJWHA, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.884309144 (238h47m29.687668328s) [noise=true , address=89.149.201.147:4023 , piece_id=NWUSBCFDTVH67AO5RFAUMRBP7I4BA6LM2SS46REMDANS5PULE33A, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.884827042 (238h47m29.68818621s) [noise=true , address=185.167.204.206:4041 , piece_id=PNMLNBEI5MCZSXIELZF22U7C5FBTJ3NKZVSMHHBZ4NHRO4CE2JMQ, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.887870537 (238h47m29.691229718s) [noise=true , address=89.149.201.147:4010 , piece_id=RY3V22SFDMDJL2SA67IVSIA3HL5ZXWFPDFEGKVTN3OJ6A3EY257Q, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.888962425 (238h47m29.692321668s) [noise=true , address=81.171.14.70:4030 , piece_id=RXX3L2JLRR63LJMPPHD4KIB4S3ETKXPJGZ7SO7YIVEPNX7J4SOUA, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:28.889823462 (238h47m29.693182639s) [noise=true , address=185.167.204.205:4016 , piece_id=TKSQKNTBVIDDBR3UPJLX7ZCNMFMEXYPXL7YJCQ4ZSPYG2H76PY3A, seg_pos={0 0}, upload=248562]: Uploading piece...
Feb 27 19:03:29.305942479 (238h47m30.109301644s) [upload=248562]: Next returned nil segment
Feb 27 19:03:29.305967164 (238h47m30.109326339s) [upload=248562]: Waiting for error group managing segments...
Feb 27 19:03:29.485292528 (238h47m30.288651714s) [noise=true , address=81.171.14.70:4032 , piece_id=GVGNNAVZFDEQPI6BT6ZRDOWEDEJGTI5CRWZQDQ7ROPCYV4WU2OMQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.488867142 (238h47m30.292226317s) [noise=true , address=81.171.14.70:4042 , piece_id=JK6BODBCGQGSFSFPS4DADNUW4SLFV2ERJPWT6IZVVGD5JNBQXKQA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.488939564 (238h47m30.292298726s) [noise=true , address=81.171.14.70:4038 , piece_id=KKVAZ7XUADG7JYHVSIXKTHJDXKCUICAGRZSBZHCSPD36LW6OYT2Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.493270337 (238h47m30.29662951s) [noise=true , address=81.171.14.70:4025 , piece_id=JAKRF6PR6HKFBA536E3EQQ3WCWKDTYUXXCWG4THF5AFMACWK5H5Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.494182345 (238h47m30.297541509s) [noise=true , address=81.171.14.70:4030 , piece_id=RXX3L2JLRR63LJMPPHD4KIB4S3ETKXPJGZ7SO7YIVEPNX7J4SOUA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.495569378 (238h47m30.298928546s) [noise=true , address=81.171.14.70:4034 , piece_id=TKUZL52SXKKF4KUVCSTYIMJTTBZZ5NZ7WYQSLH4Z4DMYQD2GLWNA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.497221478 (238h47m30.300580654s) [noise=true , address=185.167.204.205:4016 , piece_id=TKSQKNTBVIDDBR3UPJLX7ZCNMFMEXYPXL7YJCQ4ZSPYG2H76PY3A, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.498481050 (238h47m30.301840223s) [noise=true , address=81.171.14.70:4047 , piece_id=XSDESMXZSX2FPVCIM6X533ZVBQ2NRCKCUSJ7UBEOCG76RCVOFFQA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.499128725 (238h47m30.302487899s) [noise=true , address=81.171.14.70:4033 , piece_id=UTG5PYLTCAKPYWHBFIEHYPQGWX7V2CWPPZUS3IHHM6IWHG2RSYIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.500770132 (238h47m30.304129311s) [noise=true , address=185.167.204.205:4011 , piece_id=BGJAKT35K6HYKSBM75R7Y527P2WEJHTWDZNVVYZOKLZOD3GWQDTA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.520282724 (238h47m30.323641898s) [noise=true , address=89.149.201.147:4024 , piece_id=SPWYXRSYXBFJVRYRX7KUN2PPIKMOJAM5PEJJMDTU7MCO52BBOGIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.520458144 (238h47m30.323817325s) [noise=true , address=89.149.201.147:4017 , piece_id=EQXPHI4WOCMNHULZT3KP4RXHHCUVM77FGPPLOQOSBODSKAE4NVRQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.528608534 (238h47m30.331967713s) [noise=true , address=185.167.204.206:4040 , piece_id=NBIGXDP7K2G2DYUTEHCSML4X5QF6DWITXNOX6G6LJCWOLUZZJWHA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.530517514 (238h47m30.333876679s) [noise=true , address=81.171.14.70:4029 , piece_id=P4PKJ6ZBWW5XJQ3ABVC5KDGVROK6TWZFEYOLJIMVO4GH4P27NY5Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.540229723 (238h47m30.343588896s) [noise=true , address=185.167.204.206:4039 , piece_id=B4BO7RY4U3QHZKNAAGUKFVLO3F7SCHPYM4TNUXBYM3UWIFC6RSIQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.549193658 (238h47m30.352552843s) [noise=true , address=185.167.204.206:4037 , piece_id=4QRWKI66B2SVG2OWNQ5MQMQO33ZLWEV6OABDFMQWDNJVHWMRTJFQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.573203429 (238h47m30.376562603s) [noise=true , address=185.167.204.205:4007 , piece_id=JSFJFE6J7T3OYZUCU6OSGBTOGTDKXWKUX2GKZBZJPC4XMVUHI2RA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.578612555 (238h47m30.381971729s) [noise=true , address=185.167.204.206:4072 , piece_id=NWCN7VZ7C77J32GXW7MUFMACI3V45J7YQPRWIVNICO5E6N6Q5DBA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.578869362 (238h47m30.382228531s) [noise=true , address=185.167.204.206:4041 , piece_id=PNMLNBEI5MCZSXIELZF22U7C5FBTJ3NKZVSMHHBZ4NHRO4CE2JMQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.585453146 (238h47m30.388812327s) [noise=true , address=89.149.201.147:4004 , piece_id=R6KLLIMXVNYBDWH2W5KCOBWCQYF7RO47TFQKPHOR3UTYY446ATQQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.585510255 (238h47m30.388869432s) [noise=true , address=185.167.204.205:4009 , piece_id=UW66SRSI27LFSDXBPXZVA4SOIXQSEDIQAGCCPNFOIDYC6CO5DNXA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.607509471 (238h47m30.41086865s) [noise=true , address=185.167.204.206:4053 , piece_id=7CEFYGFPOKLIDAZWBEPBJ43O4OVYYUF66WABXWM7RADT4OBQ4AWQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.622982646 (238h47m30.426341819s) [noise=true , address=185.167.204.206:4046 , piece_id=6ZTW47UKRKWVYGJHOBHANVWZWE6PDHRLITMD5VUNCARVASHK47AQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:29.635158349 (238h47m30.438517522s) [noise=true , address=185.167.204.205:4020 , piece_id=BRCPVT4JBAXGNYY7N547DSMTC7L2IKGG7RHT7MU7ISL2NC3ZDBCQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:03:35.525495067 (238h47m36.328854242s) [noise=true , address=185.167.204.206:4044 , piece_id=2XMMX3NSARIQBL6SHBUTLTCTN7VC5HNHW5B5VGX4SBKNJTUDBSNA, seg_pos={0 0}, upload=248562]: Done uploading piece. err: <nil>
Feb 27 19:04:28.508369911 (238h48m29.311729091s) [noise=true , address=89.149.201.147:4010 , piece_id=RY3V22SFDMDJL2SA67IVSIA3HL5ZXWFPDFEGKVTN3OJ6A3EY257Q, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 159.223.143.39:56278->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 19:04:28.508664395 (238h48m29.312023562s) [noise=true , address=89.149.201.147:4001 , piece_id=ORKT2SIHXBZWT626NROJIJGONLRAI3YOFHAFDQ7JOOEQMHNDPAPQ, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8, address:89.149.201.147:4001): protocol: read tcp 159.223.143.39:49320->89.149.201.147:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 19:04:28.508923513 (238h48m29.312282696s) [noise=true , address=89.149.201.147:4023 , piece_id=NWUSBCFDTVH67AO5RFAUMRBP7I4BA6LM2SS46REMDANS5PULE33A, seg_pos={0 0}, upload=248562]: Done uploading piece. err: ecclient: upload failed (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv, address:89.149.201.147:4023): protocol: read tcp 159.223.143.39:48854->89.149.201.147:4023: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Feb 27 19:04:28.508961365 (238h48m29.312320533s) [seg_pos={0 0}, upload=248562]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 19:04:28.611109748 (238h48m29.414468923s) [upload=248562]: Done uploading segments...
Feb 27 19:04:28.611159571 (238h48m29.414518748s) [upload=248562]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 1m0.105428852s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.429257ms remote:
"tcp:136.0.77.3:443" response headers:
X-Amz-Request-Id=18982FD9B04A065F X-Request-Id=9M1fhFBhsRi Content-Security-Policy=block-all-mixed-content Vary=Origin X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:04:25 GMT Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml Server=Storj target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/1a1fc1f9dc8b817e
FAIL (2026-02-27 19:04:25 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FD9B04A065F</RequestId><HostId></HostId></Error>"
Duration: 10.453982ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.303872ms remote:
"tcp:45.38.24.2:443" response headers:
X-Xss-Protection=1; mode=block Content-Type=application/xml Date=Fri, 27 Feb 2026 19:04:13 GMT Accept-Ranges=bytes Content-Length=284 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Amz-Request-Id=18982FD6DEF2C093 X-Request-Id=PPZQkMsALF6 target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/76c4686fe7678cd7
FAIL (2026-02-27 19:04:13 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FD6DEF2C093</RequestId><HostId></HostId></Error>"
Duration: 30.331225ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.932826ms remote:
"tcp:136.0.77.3:443" response headers:
X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Request-Id=2GXLUBrSrnV Date=Fri, 27 Feb 2026 19:04:00 GMT Content-Type=application/xml X-Amz-Request-Id=18982FD3F530A00C target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/1b8b9f4a5634996
FAIL (2026-02-27 19:04:00 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FD3F530A00C</RequestId><HostId></HostId></Error>"
Duration: 9.958721ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
12.769858ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Type=application/xml Server=Storj X-Request-Id=WzR3xQgq4cJ Content-Length=274 Content-Security-Policy=block-all-mixed-content Vary=Origin X-Amz-Request-Id=18982FD27CC4B22F X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:03:54 GMT Accept-Ranges=bytes target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/1b84115b6849905c
FAIL (2026-02-27 19:03:54 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FD27CC4B22F</RequestId><HostId></HostId></Error>"
Duration: 12.885045ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
14.967622ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=9QtXW1m66PF Date=Fri, 27 Feb 2026 19:03:42 GMT Accept-Ranges=bytes Content-Length=274 X-Amz-Request-Id=18982FCFC3340B86 X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/21f03fbbe054e0b
FAIL (2026-02-27 19:03:42 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FCFC3340B86</RequestId><HostId></HostId></Error>"
Duration: 15.004719ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
35.570238ms remote:
"tcp:45.38.24.2:443" response headers:
X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:03:37 GMT Accept-Ranges=bytes Content-Length=284 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=dK7BnYM42zD X-Amz-Request-Id=18982FCEA2B13DF8 target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/29aa29096de369ea
FAIL (2026-02-27 19:03:37 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FCEA2B13DF8</RequestId><HostId></HostId></Error>"
Duration: 35.593756ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.276454ms remote:
"tcp:136.0.77.3:443" response headers:
Vary=Origin X-Request-Id=GW8x9nLK9Wq Content-Security-Policy=block-all-mixed-content Server=Storj X-Amz-Request-Id=18982FCC368E8246 X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:03:27 GMT Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/10d87b198151d88b
FAIL (2026-02-27 19:03:27 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FCC368E8246</RequestId><HostId></HostId></Error>"
Duration: 10.299085ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
28.012927ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Type=application/xml X-Amz-Request-Id=18982FC9BC9FEA74 X-Request-Id=Y9pnRfPC7A5 Date=Fri, 27 Feb 2026 19:03:16 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Content-Length=284 target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/54033d32d9da4c11
FAIL (2026-02-27 19:03:16 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FC9BC9FEA74</RequestId><HostId></HostId></Error>"
Duration: 28.036676ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
2.087218213s trace:
https://jaeger.dev.storj.io/trace/7aa57c119569a500
FAIL (2026-02-27 19:03:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
KCFQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408102653 (238h9m9.211461838s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408567070 (238h9m9.211926247s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408707192 (238h9m9.212066378s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408968070 (238h9m9.212327244s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409367498 (238h9m9.212726677s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409377018 (238h9m9.212736185s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410004538 (238h9m9.213363725s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410656413 (238h9m9.214015608s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411331444 (238h9m9.214690672s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411885321 (238h9m9.215244506s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412446978 (238h9m9.21580616s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412960206 (238h9m9.216319389s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.833976270 (238h9m9.637335445s) [upload=247902]: Next returned nil segment
Feb 27 18:25:08.833993781 (238h9m9.637352963s) [upload=247902]: Waiting for error group managing segments...
Feb 27 18:25:09.002997616 (238h9m9.806356791s) [noise=true , address=81.171.14.70:4040 , piece_id=2MPY2M6YD4A35CALBOQBGARWAVYBWSALZU2P4JQNVTVU462KXOSQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.003988167 (238h9m9.807347334s) [noise=true , address=81.171.14.70:4037 , piece_id=O6CQWKOJPHOHZAO7PMTX74XFWNPWIDDDUE2S4NPO67A64TRGVLDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.007807120 (238h9m9.811166324s) [noise=true , address=81.171.14.70:4044 , piece_id=SX5RETWJXQ37MNWETTT3ISLFDEQMJ4LGWEWLA4SELKUG6L447YZQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.019462938 (238h9m9.822822115s) [noise=true , address=81.171.14.70:4039 , piece_id=VV7S6QSQSAMDYP2J4TWHDJNIGMTPEHG7N4737DETH4SSYKK3FKNA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.031013955 (238h9m9.834373141s) [noise=true , address=185.167.204.206:4050 , piece_id=NNXAXJUAW34ZSFMQZHSMONU2KSGJ4DNNNLGA3VCV7Q7JE4OXYUAA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.039442786 (238h9m9.84280196s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.050960211 (238h9m9.854319391s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.067324682 (238h9m9.870683857s) [noise=true , address=185.167.204.206:4067 , piece_id=LMTR6XVP77QRHIFJFPK2CCNTLFVKGCNFXBN2Z4AQQTE7YTJ2PDVA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.070489496 (238h9m9.87384866s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.071657801 (238h9m9.87501698s) [noise=true , address=185.167.204.206:4068 , piece_id=DRL3HHGWZQGLK5RWPTJFIOUERAFO3OH3FWGWCVCPLN3JAR7K7QPA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.072680314 (238h9m9.876039488s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.074685847 (238h9m9.878045033s) [noise=true , address=185.167.204.205:4008 , piece_id=CDKH6EYARWHKZIMNICX46Z3JSGBETOJEPGVGGESJ7LZZYGGBKKMQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.089811558 (238h9m9.893170735s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.098264180 (238h9m9.901623357s) [noise=true , address=185.167.204.206:4053 , piece_id=GOQ52CA2J4GSR7A5DXDDDMEDVHODA4FE5A6A2I3GMDSICFLSKCFQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101019345 (238h9m9.904378509s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101369094 (238h9m9.904728292s) [noise=true , address=185.167.204.205:4002 , piece_id=2DSKXMEVZMV5ECFAXVPHHSK3YS5LQI3LQNTGYD3DNCHKT74VGRSA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.102365507 (238h9m9.905724671s) [noise=true , address=185.167.204.206:4041 , piece_id=JYD734S7X4QLARQFX44G24S7YXDUX3GWJ4QOY5NHD2NY5XMLVVOA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.103504275 (238h9m9.90686345s) [noise=true , address=89.149.201.147:4008 , piece_id=HY4JE6TBQ3AAXVGQNXVKIMX6MNU45UMLPK2YEH4QXC4OBKPNEWNQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.106507437 (238h9m9.909866612s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.145089732 (238h9m9.948448907s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.473005969 (238h9m10.276365143s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:13.403656159 (238h9m14.207015334s) [noise=true , address=185.167.204.206:4059 , piece_id=RYP2J5M7ERW64IPIDS4YC6XSMXJJYFY2FIOHFFVJ4FMEZMFC45PQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:14.229596272 (238h9m15.032955449s) [noise=true , address=185.167.204.206:4071 , piece_id=S23HGNGGEWNCQP2TMJ3CH2HL4TQWMEYVDFHCYKFAHXBDAY4X6ILA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:17.197218248 (238h9m18.000577422s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:50.077929419 (238h9m50.881288595s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:26:08.034087443 (238h10m8.837446626s) [noise=true , address=89.149.201.147:4016 , piece_id=DRJK3ZPIZKZ44E6RPX6FGQZWPNMA7VMAIP6JXCB7IUQQ7QC5F7BQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034276836 (238h10m8.837636013s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 159.223.143.39:60508->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034741100 (238h10m8.83810028s) [noise=true , address=185.167.204.205:4017 , piece_id=CV6LFILYHABDCW25TJJ7636AOKZ36JVLJJYXBNR7V4TZW4SJ7MUA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: read tcp 159.223.143.39:58588->185.167.204.205:4017: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034770401 (238h10m8.838129567s) [seg_pos={0 0}, upload=247902]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 18:26:08.139631321 (238h10m8.942990495s) [upload=247902]: Done uploading segments...
Feb 27 18:26:08.139695418 (238h10m8.943054606s) [upload=247902]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.087243339s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
18.538097ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Type=application/xml Vary=Origin X-Request-Id=eTFQXeYKcGW Accept-Ranges=bytes Content-Length=274 Server=Storj X-Amz-Request-Id=18982FC51153B320 X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:02:56 GMT Content-Security-Policy=block-all-mixed-content target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/c02ec228df258cd
FAIL (2026-02-27 19:02:56 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FC51153B320</RequestId><HostId></HostId></Error>"
Duration: 18.700479ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
28.052961ms remote:
"tcp:45.38.24.2:443" response headers:
X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:02:44 GMT Content-Length=284 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=FHMqLPnsaDf Accept-Ranges=bytes Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18982FC23444105B target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/24e87c9a7abdec24
FAIL (2026-02-27 19:02:44 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FC23444105B</RequestId><HostId></HostId></Error>"
Duration: 28.078324ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.236079ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Type=application/xml X-Amz-Request-Id=18982FC1191F48D6 X-Request-Id=99pUd5RWqmg X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:02:39 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin Accept-Ranges=bytes target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/6d900fa284929251
FAIL (2026-02-27 19:02:39 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FC1191F48D6</RequestId><HostId></HostId></Error>"
Duration: 11.342457ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
2.159652637s trace:
https://jaeger.dev.storj.io/trace/7aa57c119569a500
FAIL (2026-02-27 19:02:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
KCFQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408102653 (238h9m9.211461838s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408567070 (238h9m9.211926247s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408707192 (238h9m9.212066378s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408968070 (238h9m9.212327244s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409367498 (238h9m9.212726677s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409377018 (238h9m9.212736185s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410004538 (238h9m9.213363725s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410656413 (238h9m9.214015608s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411331444 (238h9m9.214690672s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411885321 (238h9m9.215244506s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412446978 (238h9m9.21580616s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412960206 (238h9m9.216319389s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.833976270 (238h9m9.637335445s) [upload=247902]: Next returned nil segment
Feb 27 18:25:08.833993781 (238h9m9.637352963s) [upload=247902]: Waiting for error group managing segments...
Feb 27 18:25:09.002997616 (238h9m9.806356791s) [noise=true , address=81.171.14.70:4040 , piece_id=2MPY2M6YD4A35CALBOQBGARWAVYBWSALZU2P4JQNVTVU462KXOSQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.003988167 (238h9m9.807347334s) [noise=true , address=81.171.14.70:4037 , piece_id=O6CQWKOJPHOHZAO7PMTX74XFWNPWIDDDUE2S4NPO67A64TRGVLDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.007807120 (238h9m9.811166324s) [noise=true , address=81.171.14.70:4044 , piece_id=SX5RETWJXQ37MNWETTT3ISLFDEQMJ4LGWEWLA4SELKUG6L447YZQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.019462938 (238h9m9.822822115s) [noise=true , address=81.171.14.70:4039 , piece_id=VV7S6QSQSAMDYP2J4TWHDJNIGMTPEHG7N4737DETH4SSYKK3FKNA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.031013955 (238h9m9.834373141s) [noise=true , address=185.167.204.206:4050 , piece_id=NNXAXJUAW34ZSFMQZHSMONU2KSGJ4DNNNLGA3VCV7Q7JE4OXYUAA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.039442786 (238h9m9.84280196s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.050960211 (238h9m9.854319391s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.067324682 (238h9m9.870683857s) [noise=true , address=185.167.204.206:4067 , piece_id=LMTR6XVP77QRHIFJFPK2CCNTLFVKGCNFXBN2Z4AQQTE7YTJ2PDVA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.070489496 (238h9m9.87384866s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.071657801 (238h9m9.87501698s) [noise=true , address=185.167.204.206:4068 , piece_id=DRL3HHGWZQGLK5RWPTJFIOUERAFO3OH3FWGWCVCPLN3JAR7K7QPA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.072680314 (238h9m9.876039488s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.074685847 (238h9m9.878045033s) [noise=true , address=185.167.204.205:4008 , piece_id=CDKH6EYARWHKZIMNICX46Z3JSGBETOJEPGVGGESJ7LZZYGGBKKMQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.089811558 (238h9m9.893170735s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.098264180 (238h9m9.901623357s) [noise=true , address=185.167.204.206:4053 , piece_id=GOQ52CA2J4GSR7A5DXDDDMEDVHODA4FE5A6A2I3GMDSICFLSKCFQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101019345 (238h9m9.904378509s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101369094 (238h9m9.904728292s) [noise=true , address=185.167.204.205:4002 , piece_id=2DSKXMEVZMV5ECFAXVPHHSK3YS5LQI3LQNTGYD3DNCHKT74VGRSA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.102365507 (238h9m9.905724671s) [noise=true , address=185.167.204.206:4041 , piece_id=JYD734S7X4QLARQFX44G24S7YXDUX3GWJ4QOY5NHD2NY5XMLVVOA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.103504275 (238h9m9.90686345s) [noise=true , address=89.149.201.147:4008 , piece_id=HY4JE6TBQ3AAXVGQNXVKIMX6MNU45UMLPK2YEH4QXC4OBKPNEWNQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.106507437 (238h9m9.909866612s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.145089732 (238h9m9.948448907s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.473005969 (238h9m10.276365143s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:13.403656159 (238h9m14.207015334s) [noise=true , address=185.167.204.206:4059 , piece_id=RYP2J5M7ERW64IPIDS4YC6XSMXJJYFY2FIOHFFVJ4FMEZMFC45PQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:14.229596272 (238h9m15.032955449s) [noise=true , address=185.167.204.206:4071 , piece_id=S23HGNGGEWNCQP2TMJ3CH2HL4TQWMEYVDFHCYKFAHXBDAY4X6ILA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:17.197218248 (238h9m18.000577422s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:50.077929419 (238h9m50.881288595s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:26:08.034087443 (238h10m8.837446626s) [noise=true , address=89.149.201.147:4016 , piece_id=DRJK3ZPIZKZ44E6RPX6FGQZWPNMA7VMAIP6JXCB7IUQQ7QC5F7BQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034276836 (238h10m8.837636013s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 159.223.143.39:60508->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034741100 (238h10m8.83810028s) [noise=true , address=185.167.204.205:4017 , piece_id=CV6LFILYHABDCW25TJJ7636AOKZ36JVLJJYXBNR7V4TZW4SJ7MUA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: read tcp 159.223.143.39:58588->185.167.204.205:4017: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034770401 (238h10m8.838129567s) [seg_pos={0 0}, upload=247902]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 18:26:08.139631321 (238h10m8.942990495s) [upload=247902]: Done uploading segments...
Feb 27 18:26:08.139695418 (238h10m8.943054606s) [upload=247902]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.159673705s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.044256ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=jQJPqRHmefv Content-Length=274 X-Amz-Request-Id=18982FBE5CC946ED X-Xss-Protection=1; mode=block Date=Fri, 27 Feb 2026 19:02:28 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/aa10fba8f50f7dd
FAIL (2026-02-27 19:02:28 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FBE5CC946ED</RequestId><HostId></HostId></Error>"
Duration: 10.067329ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.643763ms remote:
"tcp:136.0.77.3:443" response headers:
Vary=Origin X-Xss-Protection=1; mode=block Content-Length=274 Content-Type=application/xml Server=Storj X-Amz-Request-Id=18982FBD902D7A65 X-Request-Id=2tmKjhMg7t3 Date=Fri, 27 Feb 2026 19:02:24 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/1973b9b8f7742ce1
FAIL (2026-02-27 19:02:24 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FBD902D7A65</RequestId><HostId></HostId></Error>"
Duration: 10.68689ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.542565ms remote:
"tcp:45.38.24.2:443" response headers:
X-Request-Id=ieFPjjmM3Nw X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=284 Server=Storj Vary=Origin Date=Fri, 27 Feb 2026 19:01:59 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Amz-Request-Id=18982FB7A3E3F566 target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/cbe10a24c190dad
FAIL (2026-02-27 19:01:59 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FB7A3E3F566</RequestId><HostId></HostId></Error>"
Duration: 30.581083ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
14.754812ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Vary=Origin X-Amz-Request-Id=18982FB6E4B3DEB1 X-Xss-Protection=1; mode=block Content-Length=274 Content-Type=application/xml Server=Storj X-Request-Id=ea1z3N6S5nT Date=Fri, 27 Feb 2026 19:01:55 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/48735e86a20f3b35
FAIL (2026-02-27 19:01:55 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FB6E4B3DEB1</RequestId><HostId></HostId></Error>"
Duration: 14.779637ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
2.057566475s trace:
https://jaeger.dev.storj.io/trace/7aa57c119569a500
FAIL (2026-02-27 19:01:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
KCFQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408102653 (238h9m9.211461838s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408567070 (238h9m9.211926247s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408707192 (238h9m9.212066378s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.408968070 (238h9m9.212327244s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409367498 (238h9m9.212726677s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.409377018 (238h9m9.212736185s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410004538 (238h9m9.213363725s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.410656413 (238h9m9.214015608s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411331444 (238h9m9.214690672s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.411885321 (238h9m9.215244506s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412446978 (238h9m9.21580616s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.412960206 (238h9m9.216319389s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Uploading piece...
Feb 27 18:25:08.833976270 (238h9m9.637335445s) [upload=247902]: Next returned nil segment
Feb 27 18:25:08.833993781 (238h9m9.637352963s) [upload=247902]: Waiting for error group managing segments...
Feb 27 18:25:09.002997616 (238h9m9.806356791s) [noise=true , address=81.171.14.70:4040 , piece_id=2MPY2M6YD4A35CALBOQBGARWAVYBWSALZU2P4JQNVTVU462KXOSQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.003988167 (238h9m9.807347334s) [noise=true , address=81.171.14.70:4037 , piece_id=O6CQWKOJPHOHZAO7PMTX74XFWNPWIDDDUE2S4NPO67A64TRGVLDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.007807120 (238h9m9.811166324s) [noise=true , address=81.171.14.70:4044 , piece_id=SX5RETWJXQ37MNWETTT3ISLFDEQMJ4LGWEWLA4SELKUG6L447YZQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.019462938 (238h9m9.822822115s) [noise=true , address=81.171.14.70:4039 , piece_id=VV7S6QSQSAMDYP2J4TWHDJNIGMTPEHG7N4737DETH4SSYKK3FKNA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.031013955 (238h9m9.834373141s) [noise=true , address=185.167.204.206:4050 , piece_id=NNXAXJUAW34ZSFMQZHSMONU2KSGJ4DNNNLGA3VCV7Q7JE4OXYUAA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.039442786 (238h9m9.84280196s) [noise=true , address=185.167.204.205:4015 , piece_id=BDAGMFUBVFEZ36P5HRXHGSVUWECNDDTT2QJ6FTYNX7LMSVP2SXDA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.050960211 (238h9m9.854319391s) [noise=true , address=81.171.14.70:4045 , piece_id=L7UHAEK27O4JZMSCYFVKZQ7GRSLYXEVMHJ4ZLMZBBJVMVPSLLZ4Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.067324682 (238h9m9.870683857s) [noise=true , address=185.167.204.206:4067 , piece_id=LMTR6XVP77QRHIFJFPK2CCNTLFVKGCNFXBN2Z4AQQTE7YTJ2PDVA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.070489496 (238h9m9.87384866s) [noise=true , address=185.167.204.205:4014 , piece_id=BARLDQMMCDLHRYMWOBXKPUIFMJDJQMMUSBTR7VVXTO23QSZDNR4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.071657801 (238h9m9.87501698s) [noise=true , address=185.167.204.206:4068 , piece_id=DRL3HHGWZQGLK5RWPTJFIOUERAFO3OH3FWGWCVCPLN3JAR7K7QPA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.072680314 (238h9m9.876039488s) [noise=true , address=185.167.204.206:4045 , piece_id=W7IVHVWDUELYXE7ZPNN7SFIZCPF4LFKDUPF55GACLTDOKJVYVB7A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.074685847 (238h9m9.878045033s) [noise=true , address=185.167.204.205:4008 , piece_id=CDKH6EYARWHKZIMNICX46Z3JSGBETOJEPGVGGESJ7LZZYGGBKKMQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.089811558 (238h9m9.893170735s) [noise=true , address=89.149.201.147:4019 , piece_id=UIBQXE4PT74V3UTDESTN5GTPOURIQZCM7EDSSMTWP67PD3TF562A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.098264180 (238h9m9.901623357s) [noise=true , address=185.167.204.206:4053 , piece_id=GOQ52CA2J4GSR7A5DXDDDMEDVHODA4FE5A6A2I3GMDSICFLSKCFQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101019345 (238h9m9.904378509s) [noise=true , address=185.167.204.205:4024 , piece_id=Z6W5YEVS5QUZSOTCZNZIOUWA6BXVH6K2AHU4ANOLSPNYUW5Y342A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.101369094 (238h9m9.904728292s) [noise=true , address=185.167.204.205:4002 , piece_id=2DSKXMEVZMV5ECFAXVPHHSK3YS5LQI3LQNTGYD3DNCHKT74VGRSA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.102365507 (238h9m9.905724671s) [noise=true , address=185.167.204.206:4041 , piece_id=JYD734S7X4QLARQFX44G24S7YXDUX3GWJ4QOY5NHD2NY5XMLVVOA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.103504275 (238h9m9.90686345s) [noise=true , address=89.149.201.147:4008 , piece_id=HY4JE6TBQ3AAXVGQNXVKIMX6MNU45UMLPK2YEH4QXC4OBKPNEWNQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.106507437 (238h9m9.909866612s) [noise=true , address=185.167.204.205:4022 , piece_id=YQ2KZ4DBITY5ARO2HWCYE3ZZ2Z7754QZTWF6PVBWVYNKSNL5NPZA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.145089732 (238h9m9.948448907s) [noise=true , address=185.167.204.205:4032 , piece_id=PO5HCCJRMTU3HRJNBNVUURHUOQRIK3A44IG7IRZ5ID4X7XA2TK4A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:09.473005969 (238h9m10.276365143s) [noise=true , address=89.149.201.147:4022 , piece_id=RXOTFX6A2YXRKFIL3A2OG6TCRT3JOYKCTVQ4OU3WT37W5XZGKS3A, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:13.403656159 (238h9m14.207015334s) [noise=true , address=185.167.204.206:4059 , piece_id=RYP2J5M7ERW64IPIDS4YC6XSMXJJYFY2FIOHFFVJ4FMEZMFC45PQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:14.229596272 (238h9m15.032955449s) [noise=true , address=185.167.204.206:4071 , piece_id=S23HGNGGEWNCQP2TMJ3CH2HL4TQWMEYVDFHCYKFAHXBDAY4X6ILA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:17.197218248 (238h9m18.000577422s) [noise=true , address=185.167.204.206:4058 , piece_id=LWJRPZ4HGZUFWIR27LRPVADDKOFJ5ZTEOGD7CXXAL4N26FOHAW7Q, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:25:50.077929419 (238h9m50.881288595s) [noise=true , address=185.167.204.205:4016 , piece_id=OKVLDG74HIMGHBJNZ6GSSYIB4TRXPVLK3TFDDADE6XMW2ARDBAJA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: <nil>
Feb 27 18:26:08.034087443 (238h10m8.837446626s) [noise=true , address=89.149.201.147:4016 , piece_id=DRJK3ZPIZKZ44E6RPX6FGQZWPNMA7VMAIP6JXCB7IUQQ7QC5F7BQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034276836 (238h10m8.837636013s) [noise=true , address=185.167.204.206:4060 , piece_id=FYLUW3ZDKCKD6IYBCPF22RRPPXKQUIDMFBNXTVIV64KSOH4R77NQ, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 159.223.143.39:60508->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034741100 (238h10m8.83810028s) [noise=true , address=185.167.204.205:4017 , piece_id=CV6LFILYHABDCW25TJJ7636AOKZ36JVLJJYXBNR7V4TZW4SJ7MUA, seg_pos={0 0}, upload=247902]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: read tcp 159.223.143.39:58588->185.167.204.205:4017: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 27 18:26:08.034770401 (238h10m8.838129567s) [seg_pos={0 0}, upload=247902]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Feb 27 18:26:08.139631321 (238h10m8.942990495s) [upload=247902]: Done uploading segments...
Feb 27 18:26:08.139695418 (238h10m8.943054606s) [upload=247902]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.057608463s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.783334ms remote:
"tcp:136.0.77.3:443" response headers:
Date=Fri, 27 Feb 2026 19:01:18 GMT Accept-Ranges=bytes Content-Length=274 X-Request-Id=fnRD55jnL2M X-Xss-Protection=1; mode=block Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=18982FAE25A77404 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/63238867333ae1de
FAIL (2026-02-27 19:01:18 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FAE25A77404</RequestId><HostId></HostId></Error>"
Duration: 9.807759ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.615149ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Vary=Origin X-Amz-Request-Id=18982FADE568702E X-Request-Id=bvWF9u8YfaJ Date=Fri, 27 Feb 2026 19:01:17 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/33d087a568627d83
FAIL (2026-02-27 19:01:17 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18982FADE568702E</RequestId><HostId></HostId></Error>"
Duration: 8.630882ms
edge
gateway
vhost
bgp
us1
|