Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...X.......................X................................] duration:
2.130589626s trace:
https://jaeger.dev.storj.io/trace/1714bb364882ae7
FAIL (2026-03-01 18:14:16 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
e=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.592990104 (285h34m51.396349289s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593281077 (285h34m51.39664026s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593467911 (285h34m51.396827097s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593988834 (285h34m51.397348013s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.594747423 (285h34m51.398106609s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.595440292 (285h34m51.398799477s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596060855 (285h34m51.399420042s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596662398 (285h34m51.400021583s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597156952 (285h34m51.400516131s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597797496 (285h34m51.401156681s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.601483903 (285h34m51.40484307s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:51.072199575 (285h34m51.87555875s) [upload=297198]: Next returned nil segment
Mar 1 17:50:51.072208489 (285h34m51.875567667s) [upload=297198]: Waiting for error group managing segments...
Mar 1 17:50:51.229699469 (285h34m52.033058643s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.240537073 (285h34m52.043896249s) [noise=true , address=81.171.14.70:4025 , piece_id=2MZEL6PW5CE7CRHCTQNASKTHML5OZSO2FYVA3JXFY3PTVD4VQ76A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.242433161 (285h34m52.045792336s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.243381421 (285h34m52.046740599s) [noise=true , address=89.149.201.147:4020 , piece_id=UV3YSVPT535D64N7NWFMZR4TOZVYIIZZKQCL7RD67QNY5ILY3WEA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247170664 (285h34m52.050529828s) [noise=true , address=89.149.201.147:4016 , piece_id=HNANZHJKSLVJMG47RVMLLZ65C2SZJDEGXHIUGS2KNVBUWU2YV3KA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247630239 (285h34m52.050989416s) [noise=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.248632442 (285h34m52.051991607s) [noise=true , address=81.171.14.70:4040 , piece_id=PZGJB2IOQUJU3PKIL2ULOHJPOCGYZDJUUFERTL6CHVSVNHTTZQQA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.252726266 (285h34m52.056085441s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.254978063 (285h34m52.058337238s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.255376592 (285h34m52.058735766s) [noise=true , address=185.167.204.205:4036 , piece_id=Z4IHUQ6J7MGYBWBGBMQ3ZZWSMWZOWCOQHSFLVSZ3QZIAS4ZEL7WA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.259636748 (285h34m52.06299593s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.260088766 (285h34m52.063447941s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.263553786 (285h34m52.066912965s) [noise=true , address=185.167.204.206:4048 , piece_id=NEIQD6ZO6HTY7P3FGNJJ66KAJTHHA4VJMTXQ7T2PQHYR7XVTM5BQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.267390737 (285h34m52.070749917s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272420023 (285h34m52.0757792s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272548287 (285h34m52.075907461s) [noise=true , address=185.167.204.205:4017 , piece_id=PDQWK2OTU6Q62D42CBZQCJ7EU723GDBSK6G6I2EHWMCSZK3NTVGA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.284057486 (285h34m52.087416664s) [noise=true , address=81.171.14.70:4034 , piece_id=FR245FOAQ7PKT32DPRK2NTDDPME3GWYLGMDK5H2ZSWMKBRUJ33AQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.301691602 (285h34m52.105050775s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305223584 (285h34m52.108582749s) [noise=true , address=185.167.204.206:4040 , piece_id=QZWVQXCAEZN4SMHUHHOSK2FPWTFX2WQQ5URFLCIZEDK5BYYG7AJQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305782158 (285h34m52.109141323s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.317913477 (285h34m52.121272652s) [noise=true , address=89.149.201.147:4003 , piece_id=2ZC3CQ5O2HRS6R6ZNTT5HVIXO5JDXARYUHHFQ27DHG5Y52WKMB3A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.333040674 (285h34m52.136399847s) [noise=true , address=185.167.204.205:4035 , piece_id=44ER7WB2NSOXK4CA7TJ5NXYLHI2PATDKXZUAPQCRTU4ACLVVZIQQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.728950565 (285h34m52.532309749s) [noise=true , address=81.171.14.70:4031 , piece_id=BFDBTBCUUMW2CHK7V3IEG7S2JZUO5656KUWV6WNZ37VBC4GTS75A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:59.254358696 (285h35m0.057717874s) [noise=true , address=185.167.204.206:4060 , piece_id=7CB4RTLMM64ADLXFQWAMWDIQA5VV6SK2XR57S5D5UY76WEPJJGKA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:47.924585950 (285h35m48.727945146s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:50.216010704 (285h35m51.019369884s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ7TMJAZRNYLA4YQZ4Z6VM7L6NIEOL7J3WJCGOLG7ZKRJQMKUA2Q, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 159.223.143.39:47352->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216038176 (285h35m51.019397363s) [noise=true , address=89.149.201.147:4005 , piece_id=EIV2IE3Y6LDBPDPK7XPHAOTHAAZGPVLZWT44ZCYNENGOEAH33RSQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 159.223.143.39:44402->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216336113 (285h35m51.019695296s) [noise=true , address=89.149.201.147:4011 , piece_id=C2UZZPXEV36KGKHIKWFIEWEPWC7L6SP75WE3CKOKP2HG7N5QG5VA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 159.223.143.39:44042->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216380576 (285h35m51.019739756s) [seg_pos={0 0}, upload=297198]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 1 17:51:50.313652312 (285h35m51.117011496s) [upload=297198]: Done uploading segments...
Mar 1 17:51:50.313705036 (285h35m51.117064217s) [upload=297198]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.130612086s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
28.953877ms remote:
"tcp:45.38.24.2:443" response headers:
Date=Sun, 01 Mar 2026 18:14:08 GMT Content-Length=284 Content-Type=application/xml Vary=Origin X-Request-Id=eUHvNhW2H4T X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Server=Storj X-Amz-Request-Id=1898CA447E38C42E target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/64a5d5e8f158bf74
FAIL (2026-03-01 18:14:08 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA447E38C42E</RequestId><HostId></HostId></Error>"
Duration: 28.983074ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
13.342295ms remote:
"tcp:136.0.77.3:443" response headers:
Date=Sun, 01 Mar 2026 18:14:01 GMT Server=Storj Vary=Origin X-Request-Id=JT1bE2Xk6j8 X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Amz-Request-Id=1898CA42E1750238 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/3504c98baadfad9e
FAIL (2026-03-01 18:14:01 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA42E1750238</RequestId><HostId></HostId></Error>"
Duration: 13.370411ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [....X.......................X...............................] duration:
2.516467277s trace:
https://jaeger.dev.storj.io/trace/1714bb364882ae7
FAIL (2026-03-01 18:13:49 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
e=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.592990104 (285h34m51.396349289s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593281077 (285h34m51.39664026s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593467911 (285h34m51.396827097s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593988834 (285h34m51.397348013s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.594747423 (285h34m51.398106609s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.595440292 (285h34m51.398799477s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596060855 (285h34m51.399420042s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596662398 (285h34m51.400021583s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597156952 (285h34m51.400516131s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597797496 (285h34m51.401156681s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.601483903 (285h34m51.40484307s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:51.072199575 (285h34m51.87555875s) [upload=297198]: Next returned nil segment
Mar 1 17:50:51.072208489 (285h34m51.875567667s) [upload=297198]: Waiting for error group managing segments...
Mar 1 17:50:51.229699469 (285h34m52.033058643s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.240537073 (285h34m52.043896249s) [noise=true , address=81.171.14.70:4025 , piece_id=2MZEL6PW5CE7CRHCTQNASKTHML5OZSO2FYVA3JXFY3PTVD4VQ76A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.242433161 (285h34m52.045792336s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.243381421 (285h34m52.046740599s) [noise=true , address=89.149.201.147:4020 , piece_id=UV3YSVPT535D64N7NWFMZR4TOZVYIIZZKQCL7RD67QNY5ILY3WEA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247170664 (285h34m52.050529828s) [noise=true , address=89.149.201.147:4016 , piece_id=HNANZHJKSLVJMG47RVMLLZ65C2SZJDEGXHIUGS2KNVBUWU2YV3KA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247630239 (285h34m52.050989416s) [noise=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.248632442 (285h34m52.051991607s) [noise=true , address=81.171.14.70:4040 , piece_id=PZGJB2IOQUJU3PKIL2ULOHJPOCGYZDJUUFERTL6CHVSVNHTTZQQA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.252726266 (285h34m52.056085441s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.254978063 (285h34m52.058337238s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.255376592 (285h34m52.058735766s) [noise=true , address=185.167.204.205:4036 , piece_id=Z4IHUQ6J7MGYBWBGBMQ3ZZWSMWZOWCOQHSFLVSZ3QZIAS4ZEL7WA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.259636748 (285h34m52.06299593s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.260088766 (285h34m52.063447941s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.263553786 (285h34m52.066912965s) [noise=true , address=185.167.204.206:4048 , piece_id=NEIQD6ZO6HTY7P3FGNJJ66KAJTHHA4VJMTXQ7T2PQHYR7XVTM5BQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.267390737 (285h34m52.070749917s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272420023 (285h34m52.0757792s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272548287 (285h34m52.075907461s) [noise=true , address=185.167.204.205:4017 , piece_id=PDQWK2OTU6Q62D42CBZQCJ7EU723GDBSK6G6I2EHWMCSZK3NTVGA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.284057486 (285h34m52.087416664s) [noise=true , address=81.171.14.70:4034 , piece_id=FR245FOAQ7PKT32DPRK2NTDDPME3GWYLGMDK5H2ZSWMKBRUJ33AQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.301691602 (285h34m52.105050775s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305223584 (285h34m52.108582749s) [noise=true , address=185.167.204.206:4040 , piece_id=QZWVQXCAEZN4SMHUHHOSK2FPWTFX2WQQ5URFLCIZEDK5BYYG7AJQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305782158 (285h34m52.109141323s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.317913477 (285h34m52.121272652s) [noise=true , address=89.149.201.147:4003 , piece_id=2ZC3CQ5O2HRS6R6ZNTT5HVIXO5JDXARYUHHFQ27DHG5Y52WKMB3A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.333040674 (285h34m52.136399847s) [noise=true , address=185.167.204.205:4035 , piece_id=44ER7WB2NSOXK4CA7TJ5NXYLHI2PATDKXZUAPQCRTU4ACLVVZIQQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.728950565 (285h34m52.532309749s) [noise=true , address=81.171.14.70:4031 , piece_id=BFDBTBCUUMW2CHK7V3IEG7S2JZUO5656KUWV6WNZ37VBC4GTS75A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:59.254358696 (285h35m0.057717874s) [noise=true , address=185.167.204.206:4060 , piece_id=7CB4RTLMM64ADLXFQWAMWDIQA5VV6SK2XR57S5D5UY76WEPJJGKA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:47.924585950 (285h35m48.727945146s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:50.216010704 (285h35m51.019369884s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ7TMJAZRNYLA4YQZ4Z6VM7L6NIEOL7J3WJCGOLG7ZKRJQMKUA2Q, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 159.223.143.39:47352->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216038176 (285h35m51.019397363s) [noise=true , address=89.149.201.147:4005 , piece_id=EIV2IE3Y6LDBPDPK7XPHAOTHAAZGPVLZWT44ZCYNENGOEAH33RSQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 159.223.143.39:44402->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216336113 (285h35m51.019695296s) [noise=true , address=89.149.201.147:4011 , piece_id=C2UZZPXEV36KGKHIKWFIEWEPWC7L6SP75WE3CKOKP2HG7N5QG5VA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 159.223.143.39:44042->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216380576 (285h35m51.019739756s) [seg_pos={0 0}, upload=297198]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 1 17:51:50.313652312 (285h35m51.117011496s) [upload=297198]: Done uploading segments...
Mar 1 17:51:50.313705036 (285h35m51.117064217s) [upload=297198]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.516495205s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.611442ms remote:
"tcp:136.0.77.3:443" response headers:
Date=Sun, 01 Mar 2026 18:13:36 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Request-Id=KN7qEt4z2ib Accept-Ranges=bytes Content-Length=274 Vary=Origin X-Amz-Request-Id=1898CA3CE81E49C2 X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/375797a3464719ef
FAIL (2026-03-01 18:13:36 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA3CE81E49C2</RequestId><HostId></HostId></Error>"
Duration: 11.643902ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.266784ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=1898CA3906786779 Date=Sun, 01 Mar 2026 18:13:19 GMT X-Request-Id=7ce4za3FdgY X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/cbbeb9815fc905
FAIL (2026-03-01 18:13:19 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA3906786779</RequestId><HostId></HostId></Error>"
Duration: 11.293851ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
18.858274ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Amz-Request-Id=1898CA38AECF5BBF X-Xss-Protection=1; mode=block Accept-Ranges=bytes Server=Storj Vary=Origin X-Request-Id=huzseqYNNyZ Date=Sun, 01 Mar 2026 18:13:18 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/1247fa323a19c1da
FAIL (2026-03-01 18:13:18 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA38AECF5BBF</RequestId><HostId></HostId></Error>"
Duration: 18.873512ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.635153ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Server=Storj Vary=Origin X-Amz-Request-Id=1898CA37C3BD2840 X-Request-Id=jnbMtUXXrhK X-Xss-Protection=1; mode=block Date=Sun, 01 Mar 2026 18:13:14 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/2307284125025eba
FAIL (2026-03-01 18:13:14 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA37C3BD2840</RequestId><HostId></HostId></Error>"
Duration: 9.669996ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
26.704246ms remote:
"tcp:45.38.24.2:443" response headers:
Accept-Ranges=bytes Content-Type=application/xml X-Amz-Request-Id=1898CA37307736CC X-Request-Id=VBDSj8FfkKx X-Xss-Protection=1; mode=block Date=Sun, 01 Mar 2026 18:13:11 GMT Content-Length=284 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/2a707d405913f93c
FAIL (2026-03-01 18:13:11 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA37307736CC</RequestId><HostId></HostId></Error>"
Duration: 26.729059ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
13.961267ms remote:
"tcp:136.0.77.3:443" response headers:
X-Xss-Protection=1; mode=block Date=Sun, 01 Mar 2026 18:12:53 GMT Content-Length=274 Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=1898CA32ED48EC07 Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content X-Request-Id=GVe6xCmVBUY target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/17e070fb1d594cf2
FAIL (2026-03-01 18:12:53 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA32ED48EC07</RequestId><HostId></HostId></Error>"
Duration: 13.980156ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.....X.......................X..............................] duration:
2.513160085s trace:
https://jaeger.dev.storj.io/trace/1714bb364882ae7
FAIL (2026-03-01 18:12:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
e=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.592990104 (285h34m51.396349289s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593281077 (285h34m51.39664026s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593467911 (285h34m51.396827097s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593988834 (285h34m51.397348013s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.594747423 (285h34m51.398106609s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.595440292 (285h34m51.398799477s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596060855 (285h34m51.399420042s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596662398 (285h34m51.400021583s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597156952 (285h34m51.400516131s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597797496 (285h34m51.401156681s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.601483903 (285h34m51.40484307s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:51.072199575 (285h34m51.87555875s) [upload=297198]: Next returned nil segment
Mar 1 17:50:51.072208489 (285h34m51.875567667s) [upload=297198]: Waiting for error group managing segments...
Mar 1 17:50:51.229699469 (285h34m52.033058643s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.240537073 (285h34m52.043896249s) [noise=true , address=81.171.14.70:4025 , piece_id=2MZEL6PW5CE7CRHCTQNASKTHML5OZSO2FYVA3JXFY3PTVD4VQ76A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.242433161 (285h34m52.045792336s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.243381421 (285h34m52.046740599s) [noise=true , address=89.149.201.147:4020 , piece_id=UV3YSVPT535D64N7NWFMZR4TOZVYIIZZKQCL7RD67QNY5ILY3WEA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247170664 (285h34m52.050529828s) [noise=true , address=89.149.201.147:4016 , piece_id=HNANZHJKSLVJMG47RVMLLZ65C2SZJDEGXHIUGS2KNVBUWU2YV3KA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247630239 (285h34m52.050989416s) [noise=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.248632442 (285h34m52.051991607s) [noise=true , address=81.171.14.70:4040 , piece_id=PZGJB2IOQUJU3PKIL2ULOHJPOCGYZDJUUFERTL6CHVSVNHTTZQQA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.252726266 (285h34m52.056085441s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.254978063 (285h34m52.058337238s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.255376592 (285h34m52.058735766s) [noise=true , address=185.167.204.205:4036 , piece_id=Z4IHUQ6J7MGYBWBGBMQ3ZZWSMWZOWCOQHSFLVSZ3QZIAS4ZEL7WA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.259636748 (285h34m52.06299593s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.260088766 (285h34m52.063447941s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.263553786 (285h34m52.066912965s) [noise=true , address=185.167.204.206:4048 , piece_id=NEIQD6ZO6HTY7P3FGNJJ66KAJTHHA4VJMTXQ7T2PQHYR7XVTM5BQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.267390737 (285h34m52.070749917s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272420023 (285h34m52.0757792s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272548287 (285h34m52.075907461s) [noise=true , address=185.167.204.205:4017 , piece_id=PDQWK2OTU6Q62D42CBZQCJ7EU723GDBSK6G6I2EHWMCSZK3NTVGA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.284057486 (285h34m52.087416664s) [noise=true , address=81.171.14.70:4034 , piece_id=FR245FOAQ7PKT32DPRK2NTDDPME3GWYLGMDK5H2ZSWMKBRUJ33AQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.301691602 (285h34m52.105050775s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305223584 (285h34m52.108582749s) [noise=true , address=185.167.204.206:4040 , piece_id=QZWVQXCAEZN4SMHUHHOSK2FPWTFX2WQQ5URFLCIZEDK5BYYG7AJQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305782158 (285h34m52.109141323s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.317913477 (285h34m52.121272652s) [noise=true , address=89.149.201.147:4003 , piece_id=2ZC3CQ5O2HRS6R6ZNTT5HVIXO5JDXARYUHHFQ27DHG5Y52WKMB3A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.333040674 (285h34m52.136399847s) [noise=true , address=185.167.204.205:4035 , piece_id=44ER7WB2NSOXK4CA7TJ5NXYLHI2PATDKXZUAPQCRTU4ACLVVZIQQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.728950565 (285h34m52.532309749s) [noise=true , address=81.171.14.70:4031 , piece_id=BFDBTBCUUMW2CHK7V3IEG7S2JZUO5656KUWV6WNZ37VBC4GTS75A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:59.254358696 (285h35m0.057717874s) [noise=true , address=185.167.204.206:4060 , piece_id=7CB4RTLMM64ADLXFQWAMWDIQA5VV6SK2XR57S5D5UY76WEPJJGKA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:47.924585950 (285h35m48.727945146s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:50.216010704 (285h35m51.019369884s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ7TMJAZRNYLA4YQZ4Z6VM7L6NIEOL7J3WJCGOLG7ZKRJQMKUA2Q, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 159.223.143.39:47352->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216038176 (285h35m51.019397363s) [noise=true , address=89.149.201.147:4005 , piece_id=EIV2IE3Y6LDBPDPK7XPHAOTHAAZGPVLZWT44ZCYNENGOEAH33RSQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 159.223.143.39:44402->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216336113 (285h35m51.019695296s) [noise=true , address=89.149.201.147:4011 , piece_id=C2UZZPXEV36KGKHIKWFIEWEPWC7L6SP75WE3CKOKP2HG7N5QG5VA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 159.223.143.39:44042->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216380576 (285h35m51.019739756s) [seg_pos={0 0}, upload=297198]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 1 17:51:50.313652312 (285h35m51.117011496s) [upload=297198]: Done uploading segments...
Mar 1 17:51:50.313705036 (285h35m51.117064217s) [upload=297198]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.513232043s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
29.335216ms remote:
"tcp:45.38.24.2:443" response headers:
Server=Storj Vary=Origin X-Request-Id=9o2j11ht4bx X-Xss-Protection=1; mode=block Date=Sun, 01 Mar 2026 18:12:31 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Amz-Request-Id=1898CA2DBA4EC2F2 Accept-Ranges=bytes Content-Length=284 target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/a495cd938aacf52
FAIL (2026-03-01 18:12:31 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA2DBA4EC2F2</RequestId><HostId></HostId></Error>"
Duration: 29.357683ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.365599ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=1898CA2D40930C4E Accept-Ranges=bytes Content-Length=274 X-Request-Id=9pRckgtMU5x X-Xss-Protection=1; mode=block Date=Sun, 01 Mar 2026 18:12:29 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/40b7a6febbf420b5
FAIL (2026-03-01 18:12:29 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA2D40930C4E</RequestId><HostId></HostId></Error>"
Duration: 10.386921ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [......X.......................X.............................] duration:
2.550831393s trace:
https://jaeger.dev.storj.io/trace/1714bb364882ae7
FAIL (2026-03-01 18:12:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
e=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.592990104 (285h34m51.396349289s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593281077 (285h34m51.39664026s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593467911 (285h34m51.396827097s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593988834 (285h34m51.397348013s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.594747423 (285h34m51.398106609s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.595440292 (285h34m51.398799477s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596060855 (285h34m51.399420042s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596662398 (285h34m51.400021583s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597156952 (285h34m51.400516131s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597797496 (285h34m51.401156681s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.601483903 (285h34m51.40484307s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:51.072199575 (285h34m51.87555875s) [upload=297198]: Next returned nil segment
Mar 1 17:50:51.072208489 (285h34m51.875567667s) [upload=297198]: Waiting for error group managing segments...
Mar 1 17:50:51.229699469 (285h34m52.033058643s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.240537073 (285h34m52.043896249s) [noise=true , address=81.171.14.70:4025 , piece_id=2MZEL6PW5CE7CRHCTQNASKTHML5OZSO2FYVA3JXFY3PTVD4VQ76A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.242433161 (285h34m52.045792336s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.243381421 (285h34m52.046740599s) [noise=true , address=89.149.201.147:4020 , piece_id=UV3YSVPT535D64N7NWFMZR4TOZVYIIZZKQCL7RD67QNY5ILY3WEA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247170664 (285h34m52.050529828s) [noise=true , address=89.149.201.147:4016 , piece_id=HNANZHJKSLVJMG47RVMLLZ65C2SZJDEGXHIUGS2KNVBUWU2YV3KA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247630239 (285h34m52.050989416s) [noise=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.248632442 (285h34m52.051991607s) [noise=true , address=81.171.14.70:4040 , piece_id=PZGJB2IOQUJU3PKIL2ULOHJPOCGYZDJUUFERTL6CHVSVNHTTZQQA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.252726266 (285h34m52.056085441s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.254978063 (285h34m52.058337238s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.255376592 (285h34m52.058735766s) [noise=true , address=185.167.204.205:4036 , piece_id=Z4IHUQ6J7MGYBWBGBMQ3ZZWSMWZOWCOQHSFLVSZ3QZIAS4ZEL7WA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.259636748 (285h34m52.06299593s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.260088766 (285h34m52.063447941s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.263553786 (285h34m52.066912965s) [noise=true , address=185.167.204.206:4048 , piece_id=NEIQD6ZO6HTY7P3FGNJJ66KAJTHHA4VJMTXQ7T2PQHYR7XVTM5BQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.267390737 (285h34m52.070749917s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272420023 (285h34m52.0757792s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272548287 (285h34m52.075907461s) [noise=true , address=185.167.204.205:4017 , piece_id=PDQWK2OTU6Q62D42CBZQCJ7EU723GDBSK6G6I2EHWMCSZK3NTVGA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.284057486 (285h34m52.087416664s) [noise=true , address=81.171.14.70:4034 , piece_id=FR245FOAQ7PKT32DPRK2NTDDPME3GWYLGMDK5H2ZSWMKBRUJ33AQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.301691602 (285h34m52.105050775s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305223584 (285h34m52.108582749s) [noise=true , address=185.167.204.206:4040 , piece_id=QZWVQXCAEZN4SMHUHHOSK2FPWTFX2WQQ5URFLCIZEDK5BYYG7AJQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305782158 (285h34m52.109141323s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.317913477 (285h34m52.121272652s) [noise=true , address=89.149.201.147:4003 , piece_id=2ZC3CQ5O2HRS6R6ZNTT5HVIXO5JDXARYUHHFQ27DHG5Y52WKMB3A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.333040674 (285h34m52.136399847s) [noise=true , address=185.167.204.205:4035 , piece_id=44ER7WB2NSOXK4CA7TJ5NXYLHI2PATDKXZUAPQCRTU4ACLVVZIQQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.728950565 (285h34m52.532309749s) [noise=true , address=81.171.14.70:4031 , piece_id=BFDBTBCUUMW2CHK7V3IEG7S2JZUO5656KUWV6WNZ37VBC4GTS75A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:59.254358696 (285h35m0.057717874s) [noise=true , address=185.167.204.206:4060 , piece_id=7CB4RTLMM64ADLXFQWAMWDIQA5VV6SK2XR57S5D5UY76WEPJJGKA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:47.924585950 (285h35m48.727945146s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:50.216010704 (285h35m51.019369884s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ7TMJAZRNYLA4YQZ4Z6VM7L6NIEOL7J3WJCGOLG7ZKRJQMKUA2Q, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 159.223.143.39:47352->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216038176 (285h35m51.019397363s) [noise=true , address=89.149.201.147:4005 , piece_id=EIV2IE3Y6LDBPDPK7XPHAOTHAAZGPVLZWT44ZCYNENGOEAH33RSQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 159.223.143.39:44402->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216336113 (285h35m51.019695296s) [noise=true , address=89.149.201.147:4011 , piece_id=C2UZZPXEV36KGKHIKWFIEWEPWC7L6SP75WE3CKOKP2HG7N5QG5VA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 159.223.143.39:44042->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216380576 (285h35m51.019739756s) [seg_pos={0 0}, upload=297198]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 1 17:51:50.313652312 (285h35m51.117011496s) [upload=297198]: Done uploading segments...
Mar 1 17:51:50.313705036 (285h35m51.117064217s) [upload=297198]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.550869918s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.689324ms remote:
"tcp:136.0.77.3:443" response headers:
Date=Sun, 01 Mar 2026 18:12:09 GMT Content-Length=274 X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=1898CA28C4CF76C0 X-Request-Id=a6GYXh3ZPHk target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/309b5f03f230e1bb
FAIL (2026-03-01 18:12:09 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA28C4CF76C0</RequestId><HostId></HostId></Error>"
Duration: 11.710442ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.940379ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=bdrag1wZySh Content-Length=274 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=1898CA27FC748C31 X-Xss-Protection=1; mode=block Date=Sun, 01 Mar 2026 18:12:06 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/5e953b24f7f68f2b
FAIL (2026-03-01 18:12:06 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA27FC748C31</RequestId><HostId></HostId></Error>"
Duration: 11.962377ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
33.691002ms remote:
"tcp:136.0.77.3:443" response headers:
X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin Date=Sun, 01 Mar 2026 18:11:47 GMT Content-Length=274 X-Amz-Request-Id=1898CA23AEFAB651 X-Request-Id=MV1SXsVNh9R target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/6f5ac0812a75df9
FAIL (2026-03-01 18:11:47 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA23AEFAB651</RequestId><HostId></HostId></Error>"
Duration: 33.723125ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
18.500503ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=VSRVcJCiTqw X-Xss-Protection=1; mode=block X-Amz-Request-Id=1898CA22F289AC61 Date=Sun, 01 Mar 2026 18:11:44 GMT Accept-Ranges=bytes Content-Length=274 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/540a9b15e78ed859
FAIL (2026-03-01 18:11:44 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA22F289AC61</RequestId><HostId></HostId></Error>"
Duration: 18.70016ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.......X.......................X............................] duration:
2.487974763s trace:
https://jaeger.dev.storj.io/trace/1714bb364882ae7
FAIL (2026-03-01 18:11:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
e=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.592990104 (285h34m51.396349289s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593281077 (285h34m51.39664026s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593467911 (285h34m51.396827097s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593988834 (285h34m51.397348013s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.594747423 (285h34m51.398106609s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.595440292 (285h34m51.398799477s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596060855 (285h34m51.399420042s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596662398 (285h34m51.400021583s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597156952 (285h34m51.400516131s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597797496 (285h34m51.401156681s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.601483903 (285h34m51.40484307s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:51.072199575 (285h34m51.87555875s) [upload=297198]: Next returned nil segment
Mar 1 17:50:51.072208489 (285h34m51.875567667s) [upload=297198]: Waiting for error group managing segments...
Mar 1 17:50:51.229699469 (285h34m52.033058643s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.240537073 (285h34m52.043896249s) [noise=true , address=81.171.14.70:4025 , piece_id=2MZEL6PW5CE7CRHCTQNASKTHML5OZSO2FYVA3JXFY3PTVD4VQ76A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.242433161 (285h34m52.045792336s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.243381421 (285h34m52.046740599s) [noise=true , address=89.149.201.147:4020 , piece_id=UV3YSVPT535D64N7NWFMZR4TOZVYIIZZKQCL7RD67QNY5ILY3WEA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247170664 (285h34m52.050529828s) [noise=true , address=89.149.201.147:4016 , piece_id=HNANZHJKSLVJMG47RVMLLZ65C2SZJDEGXHIUGS2KNVBUWU2YV3KA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247630239 (285h34m52.050989416s) [noise=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.248632442 (285h34m52.051991607s) [noise=true , address=81.171.14.70:4040 , piece_id=PZGJB2IOQUJU3PKIL2ULOHJPOCGYZDJUUFERTL6CHVSVNHTTZQQA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.252726266 (285h34m52.056085441s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.254978063 (285h34m52.058337238s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.255376592 (285h34m52.058735766s) [noise=true , address=185.167.204.205:4036 , piece_id=Z4IHUQ6J7MGYBWBGBMQ3ZZWSMWZOWCOQHSFLVSZ3QZIAS4ZEL7WA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.259636748 (285h34m52.06299593s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.260088766 (285h34m52.063447941s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.263553786 (285h34m52.066912965s) [noise=true , address=185.167.204.206:4048 , piece_id=NEIQD6ZO6HTY7P3FGNJJ66KAJTHHA4VJMTXQ7T2PQHYR7XVTM5BQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.267390737 (285h34m52.070749917s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272420023 (285h34m52.0757792s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272548287 (285h34m52.075907461s) [noise=true , address=185.167.204.205:4017 , piece_id=PDQWK2OTU6Q62D42CBZQCJ7EU723GDBSK6G6I2EHWMCSZK3NTVGA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.284057486 (285h34m52.087416664s) [noise=true , address=81.171.14.70:4034 , piece_id=FR245FOAQ7PKT32DPRK2NTDDPME3GWYLGMDK5H2ZSWMKBRUJ33AQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.301691602 (285h34m52.105050775s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305223584 (285h34m52.108582749s) [noise=true , address=185.167.204.206:4040 , piece_id=QZWVQXCAEZN4SMHUHHOSK2FPWTFX2WQQ5URFLCIZEDK5BYYG7AJQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305782158 (285h34m52.109141323s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.317913477 (285h34m52.121272652s) [noise=true , address=89.149.201.147:4003 , piece_id=2ZC3CQ5O2HRS6R6ZNTT5HVIXO5JDXARYUHHFQ27DHG5Y52WKMB3A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.333040674 (285h34m52.136399847s) [noise=true , address=185.167.204.205:4035 , piece_id=44ER7WB2NSOXK4CA7TJ5NXYLHI2PATDKXZUAPQCRTU4ACLVVZIQQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.728950565 (285h34m52.532309749s) [noise=true , address=81.171.14.70:4031 , piece_id=BFDBTBCUUMW2CHK7V3IEG7S2JZUO5656KUWV6WNZ37VBC4GTS75A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:59.254358696 (285h35m0.057717874s) [noise=true , address=185.167.204.206:4060 , piece_id=7CB4RTLMM64ADLXFQWAMWDIQA5VV6SK2XR57S5D5UY76WEPJJGKA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:47.924585950 (285h35m48.727945146s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:50.216010704 (285h35m51.019369884s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ7TMJAZRNYLA4YQZ4Z6VM7L6NIEOL7J3WJCGOLG7ZKRJQMKUA2Q, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 159.223.143.39:47352->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216038176 (285h35m51.019397363s) [noise=true , address=89.149.201.147:4005 , piece_id=EIV2IE3Y6LDBPDPK7XPHAOTHAAZGPVLZWT44ZCYNENGOEAH33RSQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 159.223.143.39:44402->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216336113 (285h35m51.019695296s) [noise=true , address=89.149.201.147:4011 , piece_id=C2UZZPXEV36KGKHIKWFIEWEPWC7L6SP75WE3CKOKP2HG7N5QG5VA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 159.223.143.39:44042->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216380576 (285h35m51.019739756s) [seg_pos={0 0}, upload=297198]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 1 17:51:50.313652312 (285h35m51.117011496s) [upload=297198]: Done uploading segments...
Mar 1 17:51:50.313705036 (285h35m51.117064217s) [upload=297198]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.48800307s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.52784ms remote:
"tcp:45.38.24.2:443" response headers:
Accept-Ranges=bytes Content-Length=284 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1898CA1D7AC537FC X-Xss-Protection=1; mode=block Server=Storj X-Request-Id=JfrR5e1fHuC Date=Sun, 01 Mar 2026 18:11:21 GMT target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/2e66a9c015d97bc9
FAIL (2026-03-01 18:11:21 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA1D7AC537FC</RequestId><HostId></HostId></Error>"
Duration: 30.549894ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.006653ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Type=application/xml Server=Storj X-Amz-Request-Id=1898CA1CCD178ADE X-Request-Id=dJAhTWfntoM Content-Length=274 Content-Security-Policy=block-all-mixed-content Vary=Origin X-Xss-Protection=1; mode=block Date=Sun, 01 Mar 2026 18:11:18 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/44a2a8bdb20e4b4f
FAIL (2026-03-01 18:11:18 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA1CCD178ADE</RequestId><HostId></HostId></Error>"
Duration: 11.028854ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
12.511579ms remote:
"tcp:136.0.77.3:443" response headers:
X-Xss-Protection=1; mode=block Date=Sun, 01 Mar 2026 18:11:12 GMT Content-Length=274 Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=NHkhWHVtyYz Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=1898CA1B61DC760E target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/6247fcba8254b557
FAIL (2026-03-01 18:11:12 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA1B61DC760E</RequestId><HostId></HostId></Error>"
Duration: 12.532174ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
28.474519ms remote:
"tcp:45.38.24.2:443" response headers:
Accept-Ranges=bytes Content-Length=284 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=1898CA17DA49CC75 X-Request-Id=bgfzBLzxnBG X-Xss-Protection=1; mode=block Date=Sun, 01 Mar 2026 18:10:57 GMT target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/413352125a178184
FAIL (2026-03-01 18:10:57 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA17DA49CC75</RequestId><HostId></HostId></Error>"
Duration: 28.496464ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [........X.......................X...........................] duration:
2.089226826s trace:
https://jaeger.dev.storj.io/trace/1714bb364882ae7
FAIL (2026-03-01 18:10:54 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
e=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.592990104 (285h34m51.396349289s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593281077 (285h34m51.39664026s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593467911 (285h34m51.396827097s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593988834 (285h34m51.397348013s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.594747423 (285h34m51.398106609s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.595440292 (285h34m51.398799477s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596060855 (285h34m51.399420042s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596662398 (285h34m51.400021583s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597156952 (285h34m51.400516131s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597797496 (285h34m51.401156681s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.601483903 (285h34m51.40484307s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:51.072199575 (285h34m51.87555875s) [upload=297198]: Next returned nil segment
Mar 1 17:50:51.072208489 (285h34m51.875567667s) [upload=297198]: Waiting for error group managing segments...
Mar 1 17:50:51.229699469 (285h34m52.033058643s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.240537073 (285h34m52.043896249s) [noise=true , address=81.171.14.70:4025 , piece_id=2MZEL6PW5CE7CRHCTQNASKTHML5OZSO2FYVA3JXFY3PTVD4VQ76A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.242433161 (285h34m52.045792336s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.243381421 (285h34m52.046740599s) [noise=true , address=89.149.201.147:4020 , piece_id=UV3YSVPT535D64N7NWFMZR4TOZVYIIZZKQCL7RD67QNY5ILY3WEA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247170664 (285h34m52.050529828s) [noise=true , address=89.149.201.147:4016 , piece_id=HNANZHJKSLVJMG47RVMLLZ65C2SZJDEGXHIUGS2KNVBUWU2YV3KA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247630239 (285h34m52.050989416s) [noise=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.248632442 (285h34m52.051991607s) [noise=true , address=81.171.14.70:4040 , piece_id=PZGJB2IOQUJU3PKIL2ULOHJPOCGYZDJUUFERTL6CHVSVNHTTZQQA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.252726266 (285h34m52.056085441s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.254978063 (285h34m52.058337238s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.255376592 (285h34m52.058735766s) [noise=true , address=185.167.204.205:4036 , piece_id=Z4IHUQ6J7MGYBWBGBMQ3ZZWSMWZOWCOQHSFLVSZ3QZIAS4ZEL7WA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.259636748 (285h34m52.06299593s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.260088766 (285h34m52.063447941s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.263553786 (285h34m52.066912965s) [noise=true , address=185.167.204.206:4048 , piece_id=NEIQD6ZO6HTY7P3FGNJJ66KAJTHHA4VJMTXQ7T2PQHYR7XVTM5BQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.267390737 (285h34m52.070749917s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272420023 (285h34m52.0757792s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272548287 (285h34m52.075907461s) [noise=true , address=185.167.204.205:4017 , piece_id=PDQWK2OTU6Q62D42CBZQCJ7EU723GDBSK6G6I2EHWMCSZK3NTVGA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.284057486 (285h34m52.087416664s) [noise=true , address=81.171.14.70:4034 , piece_id=FR245FOAQ7PKT32DPRK2NTDDPME3GWYLGMDK5H2ZSWMKBRUJ33AQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.301691602 (285h34m52.105050775s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305223584 (285h34m52.108582749s) [noise=true , address=185.167.204.206:4040 , piece_id=QZWVQXCAEZN4SMHUHHOSK2FPWTFX2WQQ5URFLCIZEDK5BYYG7AJQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305782158 (285h34m52.109141323s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.317913477 (285h34m52.121272652s) [noise=true , address=89.149.201.147:4003 , piece_id=2ZC3CQ5O2HRS6R6ZNTT5HVIXO5JDXARYUHHFQ27DHG5Y52WKMB3A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.333040674 (285h34m52.136399847s) [noise=true , address=185.167.204.205:4035 , piece_id=44ER7WB2NSOXK4CA7TJ5NXYLHI2PATDKXZUAPQCRTU4ACLVVZIQQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.728950565 (285h34m52.532309749s) [noise=true , address=81.171.14.70:4031 , piece_id=BFDBTBCUUMW2CHK7V3IEG7S2JZUO5656KUWV6WNZ37VBC4GTS75A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:59.254358696 (285h35m0.057717874s) [noise=true , address=185.167.204.206:4060 , piece_id=7CB4RTLMM64ADLXFQWAMWDIQA5VV6SK2XR57S5D5UY76WEPJJGKA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:47.924585950 (285h35m48.727945146s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:50.216010704 (285h35m51.019369884s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ7TMJAZRNYLA4YQZ4Z6VM7L6NIEOL7J3WJCGOLG7ZKRJQMKUA2Q, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 159.223.143.39:47352->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216038176 (285h35m51.019397363s) [noise=true , address=89.149.201.147:4005 , piece_id=EIV2IE3Y6LDBPDPK7XPHAOTHAAZGPVLZWT44ZCYNENGOEAH33RSQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 159.223.143.39:44402->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216336113 (285h35m51.019695296s) [noise=true , address=89.149.201.147:4011 , piece_id=C2UZZPXEV36KGKHIKWFIEWEPWC7L6SP75WE3CKOKP2HG7N5QG5VA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 159.223.143.39:44042->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216380576 (285h35m51.019739756s) [seg_pos={0 0}, upload=297198]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 1 17:51:50.313652312 (285h35m51.117011496s) [upload=297198]: Done uploading segments...
Mar 1 17:51:50.313705036 (285h35m51.117064217s) [upload=297198]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.089253718s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.835122ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Security-Policy=block-all-mixed-content X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=1898CA16FFD94862 X-Request-Id=fNt9orTgEmU Date=Sun, 01 Mar 2026 18:10:53 GMT Content-Length=274 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/3ad0ba34d2b09b70
FAIL (2026-03-01 18:10:53 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA16FFD94862</RequestId><HostId></HostId></Error>"
Duration: 11.858405ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
13.182733ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Length=274 Vary=Origin X-Amz-Request-Id=1898CA135B1536B7 X-Request-Id=DHAKjJPf8zS Date=Sun, 01 Mar 2026 18:10:37 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Xss-Protection=1; mode=block Accept-Ranges=bytes target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/5005079bba6562fe
FAIL (2026-03-01 18:10:37 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA135B1536B7</RequestId><HostId></HostId></Error>"
Duration: 13.205857ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.252746ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Type=application/xml X-Request-Id=HxoDXVMRVQs X-Xss-Protection=1; mode=block Content-Length=274 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Amz-Request-Id=1898CA11CD4C9CA3 Date=Sun, 01 Mar 2026 18:10:31 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/18faa63f3fdab6a5
FAIL (2026-03-01 18:10:31 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA11CD4C9CA3</RequestId><HostId></HostId></Error>"
Duration: 10.277906ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.903458ms remote:
"tcp:136.0.77.3:443" response headers:
Vary=Origin X-Request-Id=45eCX3Mttbk X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Amz-Request-Id=1898CA11153C1518 Date=Sun, 01 Mar 2026 18:10:28 GMT Content-Length=274 Server=Storj target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/4360ce13822797d4
FAIL (2026-03-01 18:10:28 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA11153C1518</RequestId><HostId></HostId></Error>"
Duration: 11.92342ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
29.095988ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Length=284 Content-Security-Policy=block-all-mixed-content Server=Storj Accept-Ranges=bytes Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1898CA0ECE751CCB X-Request-Id=QXVqbMsz6iZ X-Xss-Protection=1; mode=block Date=Sun, 01 Mar 2026 18:10:18 GMT target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/27b879409fee9627
FAIL (2026-03-01 18:10:18 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA0ECE751CCB</RequestId><HostId></HostId></Error>"
Duration: 29.11891ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
13.818199ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1898CA0D8F050BBC X-Request-Id=gxw4cHM1ZfU Date=Sun, 01 Mar 2026 18:10:12 GMT Accept-Ranges=bytes Content-Length=274 Server=Storj X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/4c80d9f35a5f43b1
FAIL (2026-03-01 18:10:12 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA0D8F050BBC</RequestId><HostId></HostId></Error>"
Duration: 13.846476ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
23.207437ms remote:
"tcp:136.0.77.3:443" response headers:
Date=Sun, 01 Mar 2026 18:10:07 GMT Accept-Ranges=bytes X-Xss-Protection=1; mode=block Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=1898CA0C65B6E20B X-Request-Id=4R5oyL92vNm target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/634ccca2580b3da
FAIL (2026-03-01 18:10:07 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA0C65B6E20B</RequestId><HostId></HostId></Error>"
Duration: 23.248683ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.........X.......................X..........................] duration:
2.553172994s trace:
https://jaeger.dev.storj.io/trace/1714bb364882ae7
FAIL (2026-03-01 18:10:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
e=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.592990104 (285h34m51.396349289s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593281077 (285h34m51.39664026s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593467911 (285h34m51.396827097s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593988834 (285h34m51.397348013s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.594747423 (285h34m51.398106609s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.595440292 (285h34m51.398799477s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596060855 (285h34m51.399420042s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596662398 (285h34m51.400021583s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597156952 (285h34m51.400516131s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597797496 (285h34m51.401156681s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.601483903 (285h34m51.40484307s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:51.072199575 (285h34m51.87555875s) [upload=297198]: Next returned nil segment
Mar 1 17:50:51.072208489 (285h34m51.875567667s) [upload=297198]: Waiting for error group managing segments...
Mar 1 17:50:51.229699469 (285h34m52.033058643s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.240537073 (285h34m52.043896249s) [noise=true , address=81.171.14.70:4025 , piece_id=2MZEL6PW5CE7CRHCTQNASKTHML5OZSO2FYVA3JXFY3PTVD4VQ76A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.242433161 (285h34m52.045792336s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.243381421 (285h34m52.046740599s) [noise=true , address=89.149.201.147:4020 , piece_id=UV3YSVPT535D64N7NWFMZR4TOZVYIIZZKQCL7RD67QNY5ILY3WEA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247170664 (285h34m52.050529828s) [noise=true , address=89.149.201.147:4016 , piece_id=HNANZHJKSLVJMG47RVMLLZ65C2SZJDEGXHIUGS2KNVBUWU2YV3KA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247630239 (285h34m52.050989416s) [noise=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.248632442 (285h34m52.051991607s) [noise=true , address=81.171.14.70:4040 , piece_id=PZGJB2IOQUJU3PKIL2ULOHJPOCGYZDJUUFERTL6CHVSVNHTTZQQA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.252726266 (285h34m52.056085441s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.254978063 (285h34m52.058337238s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.255376592 (285h34m52.058735766s) [noise=true , address=185.167.204.205:4036 , piece_id=Z4IHUQ6J7MGYBWBGBMQ3ZZWSMWZOWCOQHSFLVSZ3QZIAS4ZEL7WA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.259636748 (285h34m52.06299593s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.260088766 (285h34m52.063447941s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.263553786 (285h34m52.066912965s) [noise=true , address=185.167.204.206:4048 , piece_id=NEIQD6ZO6HTY7P3FGNJJ66KAJTHHA4VJMTXQ7T2PQHYR7XVTM5BQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.267390737 (285h34m52.070749917s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272420023 (285h34m52.0757792s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272548287 (285h34m52.075907461s) [noise=true , address=185.167.204.205:4017 , piece_id=PDQWK2OTU6Q62D42CBZQCJ7EU723GDBSK6G6I2EHWMCSZK3NTVGA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.284057486 (285h34m52.087416664s) [noise=true , address=81.171.14.70:4034 , piece_id=FR245FOAQ7PKT32DPRK2NTDDPME3GWYLGMDK5H2ZSWMKBRUJ33AQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.301691602 (285h34m52.105050775s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305223584 (285h34m52.108582749s) [noise=true , address=185.167.204.206:4040 , piece_id=QZWVQXCAEZN4SMHUHHOSK2FPWTFX2WQQ5URFLCIZEDK5BYYG7AJQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305782158 (285h34m52.109141323s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.317913477 (285h34m52.121272652s) [noise=true , address=89.149.201.147:4003 , piece_id=2ZC3CQ5O2HRS6R6ZNTT5HVIXO5JDXARYUHHFQ27DHG5Y52WKMB3A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.333040674 (285h34m52.136399847s) [noise=true , address=185.167.204.205:4035 , piece_id=44ER7WB2NSOXK4CA7TJ5NXYLHI2PATDKXZUAPQCRTU4ACLVVZIQQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.728950565 (285h34m52.532309749s) [noise=true , address=81.171.14.70:4031 , piece_id=BFDBTBCUUMW2CHK7V3IEG7S2JZUO5656KUWV6WNZ37VBC4GTS75A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:59.254358696 (285h35m0.057717874s) [noise=true , address=185.167.204.206:4060 , piece_id=7CB4RTLMM64ADLXFQWAMWDIQA5VV6SK2XR57S5D5UY76WEPJJGKA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:47.924585950 (285h35m48.727945146s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:50.216010704 (285h35m51.019369884s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ7TMJAZRNYLA4YQZ4Z6VM7L6NIEOL7J3WJCGOLG7ZKRJQMKUA2Q, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 159.223.143.39:47352->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216038176 (285h35m51.019397363s) [noise=true , address=89.149.201.147:4005 , piece_id=EIV2IE3Y6LDBPDPK7XPHAOTHAAZGPVLZWT44ZCYNENGOEAH33RSQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 159.223.143.39:44402->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216336113 (285h35m51.019695296s) [noise=true , address=89.149.201.147:4011 , piece_id=C2UZZPXEV36KGKHIKWFIEWEPWC7L6SP75WE3CKOKP2HG7N5QG5VA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 159.223.143.39:44042->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216380576 (285h35m51.019739756s) [seg_pos={0 0}, upload=297198]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 1 17:51:50.313652312 (285h35m51.117011496s) [upload=297198]: Done uploading segments...
Mar 1 17:51:50.313705036 (285h35m51.117064217s) [upload=297198]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.553226627s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.119024ms remote:
"tcp:136.0.77.3:443" response headers:
Vary=Origin Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml X-Amz-Request-Id=1898CA0866CE5326 X-Request-Id=gbmQo6yRyXa X-Xss-Protection=1; mode=block Date=Sun, 01 Mar 2026 18:09:50 GMT Content-Security-Policy=block-all-mixed-content Server=Storj target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/53cca02e4cd2deac
FAIL (2026-03-01 18:09:50 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA0866CE5326</RequestId><HostId></HostId></Error>"
Duration: 11.14186ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
14.391209ms remote:
"tcp:136.0.77.3:443" response headers:
Vary=Origin X-Request-Id=7kWHcYuz461 X-Xss-Protection=1; mode=block X-Amz-Request-Id=1898CA07676FEFEC Date=Sun, 01 Mar 2026 18:09:46 GMT Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/6a089f2be907ae32
FAIL (2026-03-01 18:09:46 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA07676FEFEC</RequestId><HostId></HostId></Error>"
Duration: 14.417061ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.734883ms remote:
"tcp:45.38.24.2:443" response headers:
Accept-Ranges=bytes Content-Length=284 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Amz-Request-Id=1898CA011BD87FEB Date=Sun, 01 Mar 2026 18:09:19 GMT Server=Storj Vary=Origin X-Request-Id=V9ky8puRDoy X-Xss-Protection=1; mode=block target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/5e9b9f9c97774569
FAIL (2026-03-01 18:09:19 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898CA011BD87FEB</RequestId><HostId></HostId></Error>"
Duration: 30.758599ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.037979ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Length=274 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Request-Id=KmDYxgy6HwT X-Xss-Protection=1; mode=block Date=Sun, 01 Mar 2026 18:09:13 GMT Accept-Ranges=bytes Content-Type=application/xml X-Amz-Request-Id=1898C9FFCFAB372A target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/6d6e499aff9a7a6d
FAIL (2026-03-01 18:09:13 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898C9FFCFAB372A</RequestId><HostId></HostId></Error>"
Duration: 9.065932ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..........X.......................X.........................] duration:
2.502932672s trace:
https://jaeger.dev.storj.io/trace/1714bb364882ae7
FAIL (2026-03-01 18:08:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
e=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.592990104 (285h34m51.396349289s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593281077 (285h34m51.39664026s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593467911 (285h34m51.396827097s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593988834 (285h34m51.397348013s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.594747423 (285h34m51.398106609s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.595440292 (285h34m51.398799477s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596060855 (285h34m51.399420042s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596662398 (285h34m51.400021583s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597156952 (285h34m51.400516131s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597797496 (285h34m51.401156681s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.601483903 (285h34m51.40484307s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:51.072199575 (285h34m51.87555875s) [upload=297198]: Next returned nil segment
Mar 1 17:50:51.072208489 (285h34m51.875567667s) [upload=297198]: Waiting for error group managing segments...
Mar 1 17:50:51.229699469 (285h34m52.033058643s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.240537073 (285h34m52.043896249s) [noise=true , address=81.171.14.70:4025 , piece_id=2MZEL6PW5CE7CRHCTQNASKTHML5OZSO2FYVA3JXFY3PTVD4VQ76A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.242433161 (285h34m52.045792336s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.243381421 (285h34m52.046740599s) [noise=true , address=89.149.201.147:4020 , piece_id=UV3YSVPT535D64N7NWFMZR4TOZVYIIZZKQCL7RD67QNY5ILY3WEA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247170664 (285h34m52.050529828s) [noise=true , address=89.149.201.147:4016 , piece_id=HNANZHJKSLVJMG47RVMLLZ65C2SZJDEGXHIUGS2KNVBUWU2YV3KA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247630239 (285h34m52.050989416s) [noise=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.248632442 (285h34m52.051991607s) [noise=true , address=81.171.14.70:4040 , piece_id=PZGJB2IOQUJU3PKIL2ULOHJPOCGYZDJUUFERTL6CHVSVNHTTZQQA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.252726266 (285h34m52.056085441s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.254978063 (285h34m52.058337238s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.255376592 (285h34m52.058735766s) [noise=true , address=185.167.204.205:4036 , piece_id=Z4IHUQ6J7MGYBWBGBMQ3ZZWSMWZOWCOQHSFLVSZ3QZIAS4ZEL7WA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.259636748 (285h34m52.06299593s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.260088766 (285h34m52.063447941s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.263553786 (285h34m52.066912965s) [noise=true , address=185.167.204.206:4048 , piece_id=NEIQD6ZO6HTY7P3FGNJJ66KAJTHHA4VJMTXQ7T2PQHYR7XVTM5BQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.267390737 (285h34m52.070749917s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272420023 (285h34m52.0757792s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272548287 (285h34m52.075907461s) [noise=true , address=185.167.204.205:4017 , piece_id=PDQWK2OTU6Q62D42CBZQCJ7EU723GDBSK6G6I2EHWMCSZK3NTVGA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.284057486 (285h34m52.087416664s) [noise=true , address=81.171.14.70:4034 , piece_id=FR245FOAQ7PKT32DPRK2NTDDPME3GWYLGMDK5H2ZSWMKBRUJ33AQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.301691602 (285h34m52.105050775s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305223584 (285h34m52.108582749s) [noise=true , address=185.167.204.206:4040 , piece_id=QZWVQXCAEZN4SMHUHHOSK2FPWTFX2WQQ5URFLCIZEDK5BYYG7AJQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305782158 (285h34m52.109141323s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.317913477 (285h34m52.121272652s) [noise=true , address=89.149.201.147:4003 , piece_id=2ZC3CQ5O2HRS6R6ZNTT5HVIXO5JDXARYUHHFQ27DHG5Y52WKMB3A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.333040674 (285h34m52.136399847s) [noise=true , address=185.167.204.205:4035 , piece_id=44ER7WB2NSOXK4CA7TJ5NXYLHI2PATDKXZUAPQCRTU4ACLVVZIQQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.728950565 (285h34m52.532309749s) [noise=true , address=81.171.14.70:4031 , piece_id=BFDBTBCUUMW2CHK7V3IEG7S2JZUO5656KUWV6WNZ37VBC4GTS75A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:59.254358696 (285h35m0.057717874s) [noise=true , address=185.167.204.206:4060 , piece_id=7CB4RTLMM64ADLXFQWAMWDIQA5VV6SK2XR57S5D5UY76WEPJJGKA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:47.924585950 (285h35m48.727945146s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:50.216010704 (285h35m51.019369884s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ7TMJAZRNYLA4YQZ4Z6VM7L6NIEOL7J3WJCGOLG7ZKRJQMKUA2Q, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 159.223.143.39:47352->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216038176 (285h35m51.019397363s) [noise=true , address=89.149.201.147:4005 , piece_id=EIV2IE3Y6LDBPDPK7XPHAOTHAAZGPVLZWT44ZCYNENGOEAH33RSQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 159.223.143.39:44402->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216336113 (285h35m51.019695296s) [noise=true , address=89.149.201.147:4011 , piece_id=C2UZZPXEV36KGKHIKWFIEWEPWC7L6SP75WE3CKOKP2HG7N5QG5VA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 159.223.143.39:44042->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216380576 (285h35m51.019739756s) [seg_pos={0 0}, upload=297198]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 1 17:51:50.313652312 (285h35m51.117011496s) [upload=297198]: Done uploading segments...
Mar 1 17:51:50.313705036 (285h35m51.117064217s) [upload=297198]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.502973117s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.821701ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1898C9FA15802470 X-Request-Id=beSYcyrLXr9 Date=Sun, 01 Mar 2026 18:08:49 GMT Server=Storj X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/1c34b9ca36a430c8
FAIL (2026-03-01 18:08:49 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898C9FA15802470</RequestId><HostId></HostId></Error>"
Duration: 11.842388ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.336809ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Xss-Protection=1; mode=block Date=Sun, 01 Mar 2026 18:08:43 GMT Vary=Origin X-Amz-Request-Id=1898C9F8AB205D37 X-Request-Id=PWjwnDkva8p target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/5f80721665bca54d
FAIL (2026-03-01 18:08:43 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898C9F8AB205D37</RequestId><HostId></HostId></Error>"
Duration: 9.359735ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.932575ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Length=284 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=1898C9F84168CE0C X-Request-Id=cgRRxmZHjuk Date=Sun, 01 Mar 2026 18:08:41 GMT Accept-Ranges=bytes X-Xss-Protection=1; mode=block target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/17cf21169d4ab30f
FAIL (2026-03-01 18:08:41 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898C9F84168CE0C</RequestId><HostId></HostId></Error>"
Duration: 30.965737ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
15.084942ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Vary=Origin Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Amz-Request-Id=1898C9F77FDCF70E X-Request-Id=EGAE2VxceVK X-Xss-Protection=1; mode=block Date=Sun, 01 Mar 2026 18:08:38 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/d27e6b8531a5040
FAIL (2026-03-01 18:08:38 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898C9F77FDCF70E</RequestId><HostId></HostId></Error>"
Duration: 15.110404ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...........X.......................X........................] duration:
2.556575602s trace:
https://jaeger.dev.storj.io/trace/1714bb364882ae7
FAIL (2026-03-01 18:08:31 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
e=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.592990104 (285h34m51.396349289s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593281077 (285h34m51.39664026s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593467911 (285h34m51.396827097s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593988834 (285h34m51.397348013s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.594747423 (285h34m51.398106609s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.595440292 (285h34m51.398799477s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596060855 (285h34m51.399420042s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596662398 (285h34m51.400021583s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597156952 (285h34m51.400516131s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597797496 (285h34m51.401156681s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.601483903 (285h34m51.40484307s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:51.072199575 (285h34m51.87555875s) [upload=297198]: Next returned nil segment
Mar 1 17:50:51.072208489 (285h34m51.875567667s) [upload=297198]: Waiting for error group managing segments...
Mar 1 17:50:51.229699469 (285h34m52.033058643s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.240537073 (285h34m52.043896249s) [noise=true , address=81.171.14.70:4025 , piece_id=2MZEL6PW5CE7CRHCTQNASKTHML5OZSO2FYVA3JXFY3PTVD4VQ76A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.242433161 (285h34m52.045792336s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.243381421 (285h34m52.046740599s) [noise=true , address=89.149.201.147:4020 , piece_id=UV3YSVPT535D64N7NWFMZR4TOZVYIIZZKQCL7RD67QNY5ILY3WEA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247170664 (285h34m52.050529828s) [noise=true , address=89.149.201.147:4016 , piece_id=HNANZHJKSLVJMG47RVMLLZ65C2SZJDEGXHIUGS2KNVBUWU2YV3KA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247630239 (285h34m52.050989416s) [noise=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.248632442 (285h34m52.051991607s) [noise=true , address=81.171.14.70:4040 , piece_id=PZGJB2IOQUJU3PKIL2ULOHJPOCGYZDJUUFERTL6CHVSVNHTTZQQA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.252726266 (285h34m52.056085441s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.254978063 (285h34m52.058337238s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.255376592 (285h34m52.058735766s) [noise=true , address=185.167.204.205:4036 , piece_id=Z4IHUQ6J7MGYBWBGBMQ3ZZWSMWZOWCOQHSFLVSZ3QZIAS4ZEL7WA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.259636748 (285h34m52.06299593s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.260088766 (285h34m52.063447941s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.263553786 (285h34m52.066912965s) [noise=true , address=185.167.204.206:4048 , piece_id=NEIQD6ZO6HTY7P3FGNJJ66KAJTHHA4VJMTXQ7T2PQHYR7XVTM5BQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.267390737 (285h34m52.070749917s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272420023 (285h34m52.0757792s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272548287 (285h34m52.075907461s) [noise=true , address=185.167.204.205:4017 , piece_id=PDQWK2OTU6Q62D42CBZQCJ7EU723GDBSK6G6I2EHWMCSZK3NTVGA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.284057486 (285h34m52.087416664s) [noise=true , address=81.171.14.70:4034 , piece_id=FR245FOAQ7PKT32DPRK2NTDDPME3GWYLGMDK5H2ZSWMKBRUJ33AQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.301691602 (285h34m52.105050775s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305223584 (285h34m52.108582749s) [noise=true , address=185.167.204.206:4040 , piece_id=QZWVQXCAEZN4SMHUHHOSK2FPWTFX2WQQ5URFLCIZEDK5BYYG7AJQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305782158 (285h34m52.109141323s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.317913477 (285h34m52.121272652s) [noise=true , address=89.149.201.147:4003 , piece_id=2ZC3CQ5O2HRS6R6ZNTT5HVIXO5JDXARYUHHFQ27DHG5Y52WKMB3A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.333040674 (285h34m52.136399847s) [noise=true , address=185.167.204.205:4035 , piece_id=44ER7WB2NSOXK4CA7TJ5NXYLHI2PATDKXZUAPQCRTU4ACLVVZIQQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.728950565 (285h34m52.532309749s) [noise=true , address=81.171.14.70:4031 , piece_id=BFDBTBCUUMW2CHK7V3IEG7S2JZUO5656KUWV6WNZ37VBC4GTS75A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:59.254358696 (285h35m0.057717874s) [noise=true , address=185.167.204.206:4060 , piece_id=7CB4RTLMM64ADLXFQWAMWDIQA5VV6SK2XR57S5D5UY76WEPJJGKA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:47.924585950 (285h35m48.727945146s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:50.216010704 (285h35m51.019369884s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ7TMJAZRNYLA4YQZ4Z6VM7L6NIEOL7J3WJCGOLG7ZKRJQMKUA2Q, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 159.223.143.39:47352->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216038176 (285h35m51.019397363s) [noise=true , address=89.149.201.147:4005 , piece_id=EIV2IE3Y6LDBPDPK7XPHAOTHAAZGPVLZWT44ZCYNENGOEAH33RSQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 159.223.143.39:44402->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216336113 (285h35m51.019695296s) [noise=true , address=89.149.201.147:4011 , piece_id=C2UZZPXEV36KGKHIKWFIEWEPWC7L6SP75WE3CKOKP2HG7N5QG5VA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 159.223.143.39:44042->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216380576 (285h35m51.019739756s) [seg_pos={0 0}, upload=297198]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 1 17:51:50.313652312 (285h35m51.117011496s) [upload=297198]: Done uploading segments...
Mar 1 17:51:50.313705036 (285h35m51.117064217s) [upload=297198]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.556599488s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
29.692704ms remote:
"tcp:45.38.24.2:443" response headers:
Server=Storj X-Request-Id=Am6FfC9Nh9p Date=Sun, 01 Mar 2026 18:08:06 GMT Content-Security-Policy=block-all-mixed-content Vary=Origin X-Amz-Request-Id=1898C9F030EB233A X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=284 Content-Type=application/xml target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/3d5fb9f3ec381480
FAIL (2026-03-01 18:08:06 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898C9F030EB233A</RequestId><HostId></HostId></Error>"
Duration: 29.722829ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.149605ms remote:
"tcp:136.0.77.3:443" response headers:
Date=Sun, 01 Mar 2026 18:08:05 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Amz-Request-Id=1898C9EFE39EDE49 X-Request-Id=FRkh4yM5LLU Accept-Ranges=bytes Content-Length=274 Server=Storj Vary=Origin X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/32bcdb793ecf06b4
FAIL (2026-03-01 18:08:05 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898C9EFE39EDE49</RequestId><HostId></HostId></Error>"
Duration: 9.171445ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.035459ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Request-Id=fhd2Y6oNW9a X-Xss-Protection=1; mode=block Date=Sun, 01 Mar 2026 18:07:53 GMT Content-Length=274 Server=Storj X-Amz-Request-Id=1898C9ED0D043C5C target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/e04016ae7d1061f
FAIL (2026-03-01 18:07:53 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898C9ED0D043C5C</RequestId><HostId></HostId></Error>"
Duration: 11.058683ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
15.335408ms remote:
"tcp:136.0.77.3:443" response headers:
Server=Storj X-Amz-Request-Id=1898C9EB2EFF13EE Date=Sun, 01 Mar 2026 18:07:45 GMT Accept-Ranges=bytes Vary=Origin X-Request-Id=GuR25Cf1ohH X-Xss-Protection=1; mode=block Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/632ed7c853537677
FAIL (2026-03-01 18:07:45 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898C9EB2EFF13EE</RequestId><HostId></HostId></Error>"
Duration: 15.491519ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
13.311402ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=1898C9EAE83A3F53 Date=Sun, 01 Mar 2026 18:07:44 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content X-Request-Id=4HZMqZavRjN X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/3d739c7bec721677
FAIL (2026-03-01 18:07:44 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898C9EAE83A3F53</RequestId><HostId></HostId></Error>"
Duration: 13.347619ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.821972ms remote:
"tcp:45.38.24.2:443" response headers:
X-Amz-Request-Id=1898C9E72E39323F X-Request-Id=iH6bbvFbdmb Date=Sun, 01 Mar 2026 18:07:28 GMT Accept-Ranges=bytes Content-Length=284 X-Xss-Protection=1; mode=block Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/72d6cd9c26495e6a
FAIL (2026-03-01 18:07:28 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898C9E72E39323F</RequestId><HostId></HostId></Error>"
Duration: 30.846202ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [............X.......................X.......................] duration:
2.102828821s trace:
https://jaeger.dev.storj.io/trace/1714bb364882ae7
FAIL (2026-03-01 18:07:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
e=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.592990104 (285h34m51.396349289s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593281077 (285h34m51.39664026s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593467911 (285h34m51.396827097s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.593988834 (285h34m51.397348013s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.594747423 (285h34m51.398106609s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.595440292 (285h34m51.398799477s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596060855 (285h34m51.399420042s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.596662398 (285h34m51.400021583s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597156952 (285h34m51.400516131s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.597797496 (285h34m51.401156681s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:50.601483903 (285h34m51.40484307s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Uploading piece...
Mar 1 17:50:51.072199575 (285h34m51.87555875s) [upload=297198]: Next returned nil segment
Mar 1 17:50:51.072208489 (285h34m51.875567667s) [upload=297198]: Waiting for error group managing segments...
Mar 1 17:50:51.229699469 (285h34m52.033058643s) [noise=true , address=81.171.14.70:4046 , piece_id=MDMDIS4HFBI2AON44DXPEUHHWFY3SVJOOWKSLZLH2UVNN7HHHIOA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.240537073 (285h34m52.043896249s) [noise=true , address=81.171.14.70:4025 , piece_id=2MZEL6PW5CE7CRHCTQNASKTHML5OZSO2FYVA3JXFY3PTVD4VQ76A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.242433161 (285h34m52.045792336s) [noise=true , address=81.171.14.70:4033 , piece_id=2SVSM45QSCNEOHH5BGETUMEV7FTBWHB6K3XNY7LIC3EGTBYJDOTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.243381421 (285h34m52.046740599s) [noise=true , address=89.149.201.147:4020 , piece_id=UV3YSVPT535D64N7NWFMZR4TOZVYIIZZKQCL7RD67QNY5ILY3WEA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247170664 (285h34m52.050529828s) [noise=true , address=89.149.201.147:4016 , piece_id=HNANZHJKSLVJMG47RVMLLZ65C2SZJDEGXHIUGS2KNVBUWU2YV3KA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.247630239 (285h34m52.050989416s) [noise=true , address=81.171.14.70:4045 , piece_id=WYE2KNDLP2NWQ2WHBWT27A3GEGBYHUV6PBFAUZGSLH2RLJA7SYTQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.248632442 (285h34m52.051991607s) [noise=true , address=81.171.14.70:4040 , piece_id=PZGJB2IOQUJU3PKIL2ULOHJPOCGYZDJUUFERTL6CHVSVNHTTZQQA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.252726266 (285h34m52.056085441s) [noise=true , address=185.167.204.206:4070 , piece_id=66BX5H5H5FWEHGMGSGM34MRMMQLNTWG54WOIJUHP64UNYPXEEV7A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.254978063 (285h34m52.058337238s) [noise=true , address=89.149.201.147:4008 , piece_id=HK2UPAKQUEV277DATFAXRMVJ2I5FGYEBAAJTVZZBLNB5XY24WPJA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.255376592 (285h34m52.058735766s) [noise=true , address=185.167.204.205:4036 , piece_id=Z4IHUQ6J7MGYBWBGBMQ3ZZWSMWZOWCOQHSFLVSZ3QZIAS4ZEL7WA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.259636748 (285h34m52.06299593s) [noise=true , address=81.171.14.70:4044 , piece_id=2MK6LAL4KYCC75OOLM4B7DZX7DHAOQSQB6KQ6IMUIYRDMNFD6NTA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.260088766 (285h34m52.063447941s) [noise=true , address=81.171.14.70:4032 , piece_id=TP3DJYUYZVA6X5UFWNXA6EQW7GBKV76PYUQ3NWUIMC7T5TIT7V4A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.263553786 (285h34m52.066912965s) [noise=true , address=185.167.204.206:4048 , piece_id=NEIQD6ZO6HTY7P3FGNJJ66KAJTHHA4VJMTXQ7T2PQHYR7XVTM5BQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.267390737 (285h34m52.070749917s) [noise=true , address=81.171.14.70:4038 , piece_id=GZMICSKS4LN7CTIH3RD62YHNKHG6CJWXJBAT2VLOG5WFK5F7M5NA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272420023 (285h34m52.0757792s) [noise=true , address=81.171.14.70:4030 , piece_id=RDNIKKS6AQ3DHSCHAUSM2INLDQFUQM6VHDV3L6T4DNYYMXZYUMVA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.272548287 (285h34m52.075907461s) [noise=true , address=185.167.204.205:4017 , piece_id=PDQWK2OTU6Q62D42CBZQCJ7EU723GDBSK6G6I2EHWMCSZK3NTVGA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.284057486 (285h34m52.087416664s) [noise=true , address=81.171.14.70:4034 , piece_id=FR245FOAQ7PKT32DPRK2NTDDPME3GWYLGMDK5H2ZSWMKBRUJ33AQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.301691602 (285h34m52.105050775s) [noise=true , address=185.167.204.205:4012 , piece_id=ENANBOIXWGGRVTTE7IC6LUTZXYHMA6OR6DITYD7D27IUYZNCYZYA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305223584 (285h34m52.108582749s) [noise=true , address=185.167.204.206:4040 , piece_id=QZWVQXCAEZN4SMHUHHOSK2FPWTFX2WQQ5URFLCIZEDK5BYYG7AJQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.305782158 (285h34m52.109141323s) [noise=true , address=185.167.204.205:4007 , piece_id=UOZ7X2ONKETUW7BCM53PLTLC7UYDKBM4VZLHE7XSDA2UJKY7QBZQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.317913477 (285h34m52.121272652s) [noise=true , address=89.149.201.147:4003 , piece_id=2ZC3CQ5O2HRS6R6ZNTT5HVIXO5JDXARYUHHFQ27DHG5Y52WKMB3A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.333040674 (285h34m52.136399847s) [noise=true , address=185.167.204.205:4035 , piece_id=44ER7WB2NSOXK4CA7TJ5NXYLHI2PATDKXZUAPQCRTU4ACLVVZIQQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:51.728950565 (285h34m52.532309749s) [noise=true , address=81.171.14.70:4031 , piece_id=BFDBTBCUUMW2CHK7V3IEG7S2JZUO5656KUWV6WNZ37VBC4GTS75A, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:50:59.254358696 (285h35m0.057717874s) [noise=true , address=185.167.204.206:4060 , piece_id=7CB4RTLMM64ADLXFQWAMWDIQA5VV6SK2XR57S5D5UY76WEPJJGKA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:47.924585950 (285h35m48.727945146s) [noise=true , address=185.167.204.205:4019 , piece_id=H6BYYQWIV62UEA7MVULYZ72COHMSFWJKGTB36T4DDC7JWWAD3EYQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: <nil>
Mar 1 17:51:50.216010704 (285h35m51.019369884s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ7TMJAZRNYLA4YQZ4Z6VM7L6NIEOL7J3WJCGOLG7ZKRJQMKUA2Q, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 159.223.143.39:47352->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216038176 (285h35m51.019397363s) [noise=true , address=89.149.201.147:4005 , piece_id=EIV2IE3Y6LDBPDPK7XPHAOTHAAZGPVLZWT44ZCYNENGOEAH33RSQ, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 159.223.143.39:44402->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216336113 (285h35m51.019695296s) [noise=true , address=89.149.201.147:4011 , piece_id=C2UZZPXEV36KGKHIKWFIEWEPWC7L6SP75WE3CKOKP2HG7N5QG5VA, seg_pos={0 0}, upload=297198]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 159.223.143.39:44042->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 1 17:51:50.216380576 (285h35m51.019739756s) [seg_pos={0 0}, upload=297198]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 1 17:51:50.313652312 (285h35m51.117011496s) [upload=297198]: Done uploading segments...
Mar 1 17:51:50.313705036 (285h35m51.117064217s) [upload=297198]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.10284412s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.526629ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Type=application/xml Server=Storj Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Vary=Origin X-Amz-Request-Id=1898C9DE7BE8369A X-Request-Id=XVNPRWHR2Nn X-Xss-Protection=1; mode=block Date=Sun, 01 Mar 2026 18:06:50 GMT Content-Length=274 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/3dc41b10f52f71e0
FAIL (2026-03-01 18:06:50 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1898C9DE7BE8369A</RequestId><HostId></HostId></Error>"
Duration: 10.601157ms
edge
gateway
vhost
bgp
eu1
|