Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
1.385550664s trace:
https://jaeger.dev.storj.io/trace/14d851485ec55b94
FAIL (2026-03-10 18:38:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
egments...
Mar 10 18:34:13.234105804 (103h34m10.830876822s) [upload=115928]: Got next segment. Inline: false
Mar 10 18:34:13.234171800 (103h34m10.830942823s) [upload=115928]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 10 18:34:13.234177690 (103h34m10.830948712s) [upload=115928]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 10 18:34:13.264565602 (103h34m10.861336626s) [seg_pos={0 0}, upload=115928]: Begin upload segment...
Mar 10 18:34:13.264650554 (103h34m10.861421568s) [seg_pos={0 0}, upload=115928]: Done begin upload segment.
Mar 10 18:34:13.264710110 (103h34m10.861481134s) [noise=true , address=45.91.116.9:4280 , piece_id=O6XIUDM5QEPO2S3TSGBRWFJOA6YNOPISRIKOKJMYIYVCY3W3O6FA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.265998754 (103h34m10.862769779s) [noise=true , address=77.90.31.14:4669 , piece_id=P7P7VFJ2CX2EXILR5AVF6ZQJSRWJ7HFBWB6APJUKQOETOPO7N3CQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.266563685 (103h34m10.863334713s) [noise=true , address=77.90.31.19:4863 , piece_id=DWBZ64GZUGWORUXIZGZVHTWA3VAB4GILLMMXVU5KB2VBGRX7FN2Q, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.266980797 (103h34m10.863751825s) [noise=true , address=193.149.164.68:4044 , piece_id=HBSJYACUD45I5CQBQ3PJZOWN4MBTXJCTGHMKHOMTUG5YCDISGPOA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.267514292 (103h34m10.864285328s) [noise=true , address=79.127.205.235:4262 , piece_id=X6XAWZ64BAJP2EYT4PXLQHP6NTPCMRXSGHPTDF55V4Z3XXUZVO7A, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.267576292 (103h34m10.864347316s) [noise=true , address=193.149.164.69:4071 , piece_id=WVYSHF5Z35Z4DFCEVWVBKRRDLELXAV5URWMSFHNKCYCBQBKXBXIA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.267944808 (103h34m10.864715836s) [noise=true , address=193.149.164.67:4032 , piece_id=OYVBK5VQX2ICUTG6LTGB5DOEUKWA2Q42ANILPBUA3ELWA5XZ7OSQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.268291059 (103h34m10.865062093s) [noise=true , address=79.127.205.234:4190 , piece_id=2DCWY6MVVYBU7R5R2G6LABDDL6YCBCYEQI3B5GL2P67SFCFU7ECQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.268635689 (103h34m10.865406733s) [noise=true , address=79.127.205.237:4241 , piece_id=EMX5GFPZX3VLRD4AG3SDATNVA3YSJ6EAB4WXY3LTFE6AWEO7XZ4A, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.268857349 (103h34m10.865628383s) [noise=true , address=77.90.31.11:4587 , piece_id=XDEV3CN5NW3VMYC7B3HYF2YLSQ6GV334LG2YYD2X4PIJQE3ZNEYA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.269379163 (103h34m10.866150189s) [noise=true , address=145.79.227.31:4822 , piece_id=7A2BFMPYL6EWUQC5RFZY6IIVXIVQLML6TSZVUP7VU4NIQLWYB63Q, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.269442067 (103h34m10.866213117s) [noise=true , address=77.90.31.13:4647 , piece_id=5DBAZJVJSJZF2AHFPAQBVTDKTF4LRXTJRNNLMSNDHD6G5TY366GQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.270023291 (103h34m10.866794323s) [noise=true , address=193.149.164.71:4298 , piece_id=VMRPXXIJIKZAWY6SCIA3FMAFIQ4XYI4LWXLVFRNMSKY5B6MV5DMQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.270558766 (103h34m10.867329801s) [noise=true , address=79.127.219.41:4954 , piece_id=UMD2OKNI36MLUAC5TOF6PVK573GN3JJSM3HXCYOLG25EGRPDVHJA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.271066172 (103h34m10.867837205s) [noise=true , address=77.90.31.18:4837 , piece_id=N5RMIIIGCAQES6G6YXHCUX2YRVKIE764F57XQFGWOCKYOBJV33ZA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.271143045 (103h34m10.867914074s) [noise=true , address=167.17.72.68:4645 , piece_id=76HO76ZYEW7YE2XGOZY3EJJHTZFW24B7IFGF3IMUV6IMYKTHLG2A, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.271561296 (103h34m10.86833233s) [noise=true , address=167.17.72.70:4755 , piece_id=NXGZWBH2UNVNSE7B2BHVBEC434N7MWHDYTDECVTZNB3O6GYPZUNA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.272067991 (103h34m10.868839022s) [noise=true , address=79.127.219.45:4878 , piece_id=GTM5BKXU6TQKMBCF5FOW34OKWVQLUJUJ4WINDYBZFR3ORI2OSHSA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.341870072 (103h34m10.938641097s) [upload=115928]: Next returned nil segment
Mar 10 18:34:13.341887354 (103h34m10.938658381s) [upload=115928]: Waiting for error group managing segments...
Mar 10 18:34:13.357317608 (103h34m10.954088641s) [noise=true , address=79.127.219.41:4954 , piece_id=UMD2OKNI36MLUAC5TOF6PVK573GN3JJSM3HXCYOLG25EGRPDVHJA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.369079638 (103h34m10.965850664s) [noise=true , address=45.91.116.9:4280 , piece_id=O6XIUDM5QEPO2S3TSGBRWFJOA6YNOPISRIKOKJMYIYVCY3W3O6FA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.383782380 (103h34m10.980553407s) [noise=true , address=145.79.227.31:4822 , piece_id=7A2BFMPYL6EWUQC5RFZY6IIVXIVQLML6TSZVUP7VU4NIQLWYB63Q, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.383844707 (103h34m10.980615732s) [noise=true , address=79.127.219.45:4878 , piece_id=GTM5BKXU6TQKMBCF5FOW34OKWVQLUJUJ4WINDYBZFR3ORI2OSHSA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.596292826 (103h34m11.19306384s) [noise=true , address=167.17.72.68:4645 , piece_id=76HO76ZYEW7YE2XGOZY3EJJHTZFW24B7IFGF3IMUV6IMYKTHLG2A, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.657877077 (103h34m11.254648103s) [noise=true , address=167.17.72.70:4755 , piece_id=NXGZWBH2UNVNSE7B2BHVBEC434N7MWHDYTDECVTZNB3O6GYPZUNA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.815668472 (103h34m11.412439501s) [noise=true , address=79.127.205.234:4190 , piece_id=2DCWY6MVVYBU7R5R2G6LABDDL6YCBCYEQI3B5GL2P67SFCFU7ECQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.816829420 (103h34m11.413600446s) [noise=true , address=79.127.205.237:4241 , piece_id=EMX5GFPZX3VLRD4AG3SDATNVA3YSJ6EAB4WXY3LTFE6AWEO7XZ4A, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.818063674 (103h34m11.414834697s) [noise=true , address=79.127.205.235:4262 , piece_id=X6XAWZ64BAJP2EYT4PXLQHP6NTPCMRXSGHPTDF55V4Z3XXUZVO7A, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.844922561 (103h34m11.441693585s) [noise=true , address=193.149.164.67:4032 , piece_id=OYVBK5VQX2ICUTG6LTGB5DOEUKWA2Q42ANILPBUA3ELWA5XZ7OSQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.862417635 (103h34m11.459188668s) [noise=true , address=193.149.164.71:4298 , piece_id=VMRPXXIJIKZAWY6SCIA3FMAFIQ4XYI4LWXLVFRNMSKY5B6MV5DMQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.864932308 (103h34m11.461703323s) [noise=true , address=193.149.164.69:4071 , piece_id=WVYSHF5Z35Z4DFCEVWVBKRRDLELXAV5URWMSFHNKCYCBQBKXBXIA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.874686150 (103h34m11.471457173s) [noise=true , address=193.149.164.68:4044 , piece_id=HBSJYACUD45I5CQBQ3PJZOWN4MBTXJCTGHMKHOMTUG5YCDISGPOA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:35:13.203204147 (103h35m10.799975179s) [noise=true , address=77.90.31.19:4863 , piece_id=DWBZ64GZUGWORUXIZGZVHTWA3VAB4GILLMMXVU5KB2VBGRX7FN2Q, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL, address:77.90.31.19:4863): protocol: rpc: dial tcp 77.90.31.19:4863: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.203494517 (103h35m10.800265546s) [noise=true , address=77.90.31.18:4837 , piece_id=N5RMIIIGCAQES6G6YXHCUX2YRVKIE764F57XQFGWOCKYOBJV33ZA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:12YfSnynVtsWn1XtXHGeik1vUgqzRtArmEJJrWcXd23m2yUTS8j, address:77.90.31.18:4837): protocol: write tcp 159.223.143.39:56048->77.90.31.18:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.203740816 (103h35m10.800511848s) [noise=true , address=77.90.31.13:4647 , piece_id=5DBAZJVJSJZF2AHFPAQBVTDKTF4LRXTJRNNLMSNDHD6G5TY366GQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:121XgCUbGhKnfnTDNasmpgsQzzkZAqnFie53A7pB793wKT9VCZv, address:77.90.31.13:4647): protocol: rpc: dial tcp 77.90.31.13:4647: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.203754450 (103h35m10.800525479s) [noise=true , address=77.90.31.11:4587 , piece_id=XDEV3CN5NW3VMYC7B3HYF2YLSQ6GV334LG2YYD2X4PIJQE3ZNEYA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:1twkDDcpRrmR5bFhjGKKccMxC4Y35xh7tLaFxawhBmZkVWh3Bb, address:77.90.31.11:4587): protocol: rpc: dial tcp 77.90.31.11:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.204001069 (103h35m10.800772099s) [noise=true , address=77.90.31.14:4669 , piece_id=P7P7VFJ2CX2EXILR5AVF6ZQJSRWJ7HFBWB6APJUKQOETOPO7N3CQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.204030719 (103h35m10.800801732s) [seg_pos={0 0}, upload=115928]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Mar 10 18:35:13.239310552 (103h35m10.836081578s) [upload=115928]: Done uploading segments...
Mar 10 18:35:13.239351234 (103h35m10.836122261s) [upload=115928]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 1.385575453s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.......................................X..X.................] duration:
2.572661613s trace:
https://jaeger.dev.storj.io/trace/62b857fbddd8c134
FAIL (2026-03-10 18:38:07 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
TJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.678723726 (103h22m15.275494752s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679171303 (103h22m15.275942329s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679573269 (103h22m15.276344303s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680028932 (103h22m15.27679996s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680473046 (103h22m15.277244178s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680982200 (103h22m15.277753227s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681478081 (103h22m15.27824911s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681895554 (103h22m15.278666585s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682340386 (103h22m15.279111418s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682758716 (103h22m15.279529749s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.683342542 (103h22m15.280113571s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:18.102504696 (103h22m15.699275711s) [upload=115702]: Next returned nil segment
Mar 10 18:22:18.102520582 (103h22m15.699291606s) [upload=115702]: Waiting for error group managing segments...
Mar 10 18:22:18.286488354 (103h22m15.883259367s) [noise=true , address=185.167.204.206:4037 , piece_id=TEYUSWWSWXEKS2TUKAXH7NXD57TQKEZLPS5IOYQVC54ZTY2LEHGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.291859985 (103h22m15.888631012s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.297964517 (103h22m15.894735561s) [noise=true , address=89.149.201.147:4005 , piece_id=CFTY5WE3SZLKOPH6KT4ZD3M2A7ZCIOH7BCUUPGTYM6NRFNJC2BZA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.307699767 (103h22m15.904470781s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.312994778 (103h22m15.909765791s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.319374438 (103h22m15.916145452s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.341507663 (103h22m15.938278696s) [noise=true , address=185.167.204.206:4054 , piece_id=GCJ4ZXR6YMUGI5BG2XOYP4FKTTTSPGVZ5UYS4BOMALGHCSBFQO7Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.351324302 (103h22m15.948095328s) [noise=true , address=81.171.14.70:4032 , piece_id=GPOUM37IH7EL7YANALUZPAQOQ2MHAFOZSVWM7VLDPCNRODOKOO5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.354375553 (103h22m15.951146587s) [noise=true , address=185.167.204.205:4032 , piece_id=YA3TR364FWAMYKQUROV4PYQWTGIXM7SLAUM3BN2FRVBW3MVILHDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.355210000 (103h22m15.951981024s) [noise=true , address=89.149.201.147:4020 , piece_id=AGOZMT6EYQKSFRYOJW4QAV2NMSPM4Q5LAI2SA276EKYK4X5ZGP5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.364878440 (103h22m15.961649464s) [noise=true , address=185.167.204.206:4039 , piece_id=4LSO5WGH5LQ5NIDHOHOACKVAYY37JEZHAUX66MEJ4FIGZ5LWDRFA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.366270355 (103h22m15.963041377s) [noise=true , address=185.167.204.206:4049 , piece_id=JPGUJCUADQNZU5KJAHY4FXUTKJVSEEGMZQ7HNUEWOIWYBJ3LYPEQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.383218910 (103h22m15.979989919s) [noise=true , address=185.167.204.205:4025 , piece_id=2BA453RCULDYAHVTMMVFTPT5MRC4VJNBKK3IHUOLSOMERCHA6LKA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.385734313 (103h22m15.982505326s) [noise=true , address=185.167.204.206:4062 , piece_id=LOWNTJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.386456163 (103h22m15.983227177s) [noise=true , address=185.167.204.206:4071 , piece_id=XT7Y2BSAL4MSA4P6GXDWGQ3R7TGBZT4ON7KAJNVUMBHYPPYU4YTA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389060733 (103h22m15.985831747s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389075564 (103h22m15.985846595s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389787069 (103h22m15.986558092s) [noise=true , address=185.167.204.205:4013 , piece_id=ZYGKKFJQRQZZVEO7DQ67E3ILC66TO5EQBXBAFMKRS2CMKKOKLLVA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.392844303 (103h22m15.989615315s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.395040677 (103h22m15.99181169s) [noise=true , address=185.167.204.206:4053 , piece_id=D72PWJW544HPN6C4C6YIIQBUP7ZIWIN4VCTSYRLGUSCIEUHBWMQA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397224921 (103h22m15.993995937s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397305588 (103h22m15.994076606s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.423243063 (103h22m16.020014089s) [noise=true , address=185.167.204.206:4067 , piece_id=XWH6ZTD6S3TN6TIFN5S4ZIBNR44VOYCB7UCKXOMBBM2MPMNSXUGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:24.985281962 (103h22m22.582052989s) [noise=true , address=185.167.204.205:4007 , piece_id=GDYOPUY5PJFCYLU7OYPBVTW424GT7FP77TMRYKYNKUWXZS6ENAYQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:30.233899651 (103h22m27.830670686s) [noise=true , address=185.167.204.205:4020 , piece_id=5GIMRHZGF2ZEFWLBL4U6KF4DKZ53JZ7PNWB4XTEQW5XCESHNZY5A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:23:17.302478912 (103h23m14.899249945s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection
Mar 10 18:23:17.302510476 (103h23m14.899281492s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 159.223.143.39:39206->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:23:17.302719428 (103h23m14.89949046s) [noise=true , address=185.167.204.206:4058 , piece_id=LZ6PUPONPGDVBLY3IIUUFUZENZTSTWNMCBEK4GVI3Z2RQXICZSDQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: expected piece hash; context deadline exceeded; EOF
Mar 10 18:23:17.302761551 (103h23m14.899532565s) [seg_pos={0 0}, upload=115702]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:23:17.401134721 (103h23m14.997905738s) [upload=115702]: Done uploading segments...
Mar 10 18:23:17.401207170 (103h23m14.997978196s) [upload=115702]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.572691701s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...............................................X............] duration:
510.242448ms trace:
https://jaeger.dev.storj.io/trace/33e9eb6cbded500c
FAIL (2026-03-10 18:37:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 510.275944ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
635.735388ms trace:
https://jaeger.dev.storj.io/trace/62ea03c3a72aaf43
FAIL (2026-03-10 18:37:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 635.825451ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
1.498328207s trace:
https://jaeger.dev.storj.io/trace/14d851485ec55b94
FAIL (2026-03-10 18:37:35 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
egments...
Mar 10 18:34:13.234105804 (103h34m10.830876822s) [upload=115928]: Got next segment. Inline: false
Mar 10 18:34:13.234171800 (103h34m10.830942823s) [upload=115928]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 10 18:34:13.234177690 (103h34m10.830948712s) [upload=115928]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 10 18:34:13.264565602 (103h34m10.861336626s) [seg_pos={0 0}, upload=115928]: Begin upload segment...
Mar 10 18:34:13.264650554 (103h34m10.861421568s) [seg_pos={0 0}, upload=115928]: Done begin upload segment.
Mar 10 18:34:13.264710110 (103h34m10.861481134s) [noise=true , address=45.91.116.9:4280 , piece_id=O6XIUDM5QEPO2S3TSGBRWFJOA6YNOPISRIKOKJMYIYVCY3W3O6FA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.265998754 (103h34m10.862769779s) [noise=true , address=77.90.31.14:4669 , piece_id=P7P7VFJ2CX2EXILR5AVF6ZQJSRWJ7HFBWB6APJUKQOETOPO7N3CQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.266563685 (103h34m10.863334713s) [noise=true , address=77.90.31.19:4863 , piece_id=DWBZ64GZUGWORUXIZGZVHTWA3VAB4GILLMMXVU5KB2VBGRX7FN2Q, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.266980797 (103h34m10.863751825s) [noise=true , address=193.149.164.68:4044 , piece_id=HBSJYACUD45I5CQBQ3PJZOWN4MBTXJCTGHMKHOMTUG5YCDISGPOA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.267514292 (103h34m10.864285328s) [noise=true , address=79.127.205.235:4262 , piece_id=X6XAWZ64BAJP2EYT4PXLQHP6NTPCMRXSGHPTDF55V4Z3XXUZVO7A, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.267576292 (103h34m10.864347316s) [noise=true , address=193.149.164.69:4071 , piece_id=WVYSHF5Z35Z4DFCEVWVBKRRDLELXAV5URWMSFHNKCYCBQBKXBXIA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.267944808 (103h34m10.864715836s) [noise=true , address=193.149.164.67:4032 , piece_id=OYVBK5VQX2ICUTG6LTGB5DOEUKWA2Q42ANILPBUA3ELWA5XZ7OSQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.268291059 (103h34m10.865062093s) [noise=true , address=79.127.205.234:4190 , piece_id=2DCWY6MVVYBU7R5R2G6LABDDL6YCBCYEQI3B5GL2P67SFCFU7ECQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.268635689 (103h34m10.865406733s) [noise=true , address=79.127.205.237:4241 , piece_id=EMX5GFPZX3VLRD4AG3SDATNVA3YSJ6EAB4WXY3LTFE6AWEO7XZ4A, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.268857349 (103h34m10.865628383s) [noise=true , address=77.90.31.11:4587 , piece_id=XDEV3CN5NW3VMYC7B3HYF2YLSQ6GV334LG2YYD2X4PIJQE3ZNEYA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.269379163 (103h34m10.866150189s) [noise=true , address=145.79.227.31:4822 , piece_id=7A2BFMPYL6EWUQC5RFZY6IIVXIVQLML6TSZVUP7VU4NIQLWYB63Q, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.269442067 (103h34m10.866213117s) [noise=true , address=77.90.31.13:4647 , piece_id=5DBAZJVJSJZF2AHFPAQBVTDKTF4LRXTJRNNLMSNDHD6G5TY366GQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.270023291 (103h34m10.866794323s) [noise=true , address=193.149.164.71:4298 , piece_id=VMRPXXIJIKZAWY6SCIA3FMAFIQ4XYI4LWXLVFRNMSKY5B6MV5DMQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.270558766 (103h34m10.867329801s) [noise=true , address=79.127.219.41:4954 , piece_id=UMD2OKNI36MLUAC5TOF6PVK573GN3JJSM3HXCYOLG25EGRPDVHJA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.271066172 (103h34m10.867837205s) [noise=true , address=77.90.31.18:4837 , piece_id=N5RMIIIGCAQES6G6YXHCUX2YRVKIE764F57XQFGWOCKYOBJV33ZA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.271143045 (103h34m10.867914074s) [noise=true , address=167.17.72.68:4645 , piece_id=76HO76ZYEW7YE2XGOZY3EJJHTZFW24B7IFGF3IMUV6IMYKTHLG2A, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.271561296 (103h34m10.86833233s) [noise=true , address=167.17.72.70:4755 , piece_id=NXGZWBH2UNVNSE7B2BHVBEC434N7MWHDYTDECVTZNB3O6GYPZUNA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.272067991 (103h34m10.868839022s) [noise=true , address=79.127.219.45:4878 , piece_id=GTM5BKXU6TQKMBCF5FOW34OKWVQLUJUJ4WINDYBZFR3ORI2OSHSA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.341870072 (103h34m10.938641097s) [upload=115928]: Next returned nil segment
Mar 10 18:34:13.341887354 (103h34m10.938658381s) [upload=115928]: Waiting for error group managing segments...
Mar 10 18:34:13.357317608 (103h34m10.954088641s) [noise=true , address=79.127.219.41:4954 , piece_id=UMD2OKNI36MLUAC5TOF6PVK573GN3JJSM3HXCYOLG25EGRPDVHJA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.369079638 (103h34m10.965850664s) [noise=true , address=45.91.116.9:4280 , piece_id=O6XIUDM5QEPO2S3TSGBRWFJOA6YNOPISRIKOKJMYIYVCY3W3O6FA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.383782380 (103h34m10.980553407s) [noise=true , address=145.79.227.31:4822 , piece_id=7A2BFMPYL6EWUQC5RFZY6IIVXIVQLML6TSZVUP7VU4NIQLWYB63Q, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.383844707 (103h34m10.980615732s) [noise=true , address=79.127.219.45:4878 , piece_id=GTM5BKXU6TQKMBCF5FOW34OKWVQLUJUJ4WINDYBZFR3ORI2OSHSA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.596292826 (103h34m11.19306384s) [noise=true , address=167.17.72.68:4645 , piece_id=76HO76ZYEW7YE2XGOZY3EJJHTZFW24B7IFGF3IMUV6IMYKTHLG2A, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.657877077 (103h34m11.254648103s) [noise=true , address=167.17.72.70:4755 , piece_id=NXGZWBH2UNVNSE7B2BHVBEC434N7MWHDYTDECVTZNB3O6GYPZUNA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.815668472 (103h34m11.412439501s) [noise=true , address=79.127.205.234:4190 , piece_id=2DCWY6MVVYBU7R5R2G6LABDDL6YCBCYEQI3B5GL2P67SFCFU7ECQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.816829420 (103h34m11.413600446s) [noise=true , address=79.127.205.237:4241 , piece_id=EMX5GFPZX3VLRD4AG3SDATNVA3YSJ6EAB4WXY3LTFE6AWEO7XZ4A, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.818063674 (103h34m11.414834697s) [noise=true , address=79.127.205.235:4262 , piece_id=X6XAWZ64BAJP2EYT4PXLQHP6NTPCMRXSGHPTDF55V4Z3XXUZVO7A, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.844922561 (103h34m11.441693585s) [noise=true , address=193.149.164.67:4032 , piece_id=OYVBK5VQX2ICUTG6LTGB5DOEUKWA2Q42ANILPBUA3ELWA5XZ7OSQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.862417635 (103h34m11.459188668s) [noise=true , address=193.149.164.71:4298 , piece_id=VMRPXXIJIKZAWY6SCIA3FMAFIQ4XYI4LWXLVFRNMSKY5B6MV5DMQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.864932308 (103h34m11.461703323s) [noise=true , address=193.149.164.69:4071 , piece_id=WVYSHF5Z35Z4DFCEVWVBKRRDLELXAV5URWMSFHNKCYCBQBKXBXIA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.874686150 (103h34m11.471457173s) [noise=true , address=193.149.164.68:4044 , piece_id=HBSJYACUD45I5CQBQ3PJZOWN4MBTXJCTGHMKHOMTUG5YCDISGPOA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:35:13.203204147 (103h35m10.799975179s) [noise=true , address=77.90.31.19:4863 , piece_id=DWBZ64GZUGWORUXIZGZVHTWA3VAB4GILLMMXVU5KB2VBGRX7FN2Q, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL, address:77.90.31.19:4863): protocol: rpc: dial tcp 77.90.31.19:4863: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.203494517 (103h35m10.800265546s) [noise=true , address=77.90.31.18:4837 , piece_id=N5RMIIIGCAQES6G6YXHCUX2YRVKIE764F57XQFGWOCKYOBJV33ZA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:12YfSnynVtsWn1XtXHGeik1vUgqzRtArmEJJrWcXd23m2yUTS8j, address:77.90.31.18:4837): protocol: write tcp 159.223.143.39:56048->77.90.31.18:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.203740816 (103h35m10.800511848s) [noise=true , address=77.90.31.13:4647 , piece_id=5DBAZJVJSJZF2AHFPAQBVTDKTF4LRXTJRNNLMSNDHD6G5TY366GQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:121XgCUbGhKnfnTDNasmpgsQzzkZAqnFie53A7pB793wKT9VCZv, address:77.90.31.13:4647): protocol: rpc: dial tcp 77.90.31.13:4647: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.203754450 (103h35m10.800525479s) [noise=true , address=77.90.31.11:4587 , piece_id=XDEV3CN5NW3VMYC7B3HYF2YLSQ6GV334LG2YYD2X4PIJQE3ZNEYA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:1twkDDcpRrmR5bFhjGKKccMxC4Y35xh7tLaFxawhBmZkVWh3Bb, address:77.90.31.11:4587): protocol: rpc: dial tcp 77.90.31.11:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.204001069 (103h35m10.800772099s) [noise=true , address=77.90.31.14:4669 , piece_id=P7P7VFJ2CX2EXILR5AVF6ZQJSRWJ7HFBWB6APJUKQOETOPO7N3CQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.204030719 (103h35m10.800801732s) [seg_pos={0 0}, upload=115928]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Mar 10 18:35:13.239310552 (103h35m10.836081578s) [upload=115928]: Done uploading segments...
Mar 10 18:35:13.239351234 (103h35m10.836122261s) [upload=115928]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 1.498353833s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [........................................X..X................] duration:
2.564019173s trace:
https://jaeger.dev.storj.io/trace/62b857fbddd8c134
FAIL (2026-03-10 18:37:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
TJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.678723726 (103h22m15.275494752s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679171303 (103h22m15.275942329s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679573269 (103h22m15.276344303s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680028932 (103h22m15.27679996s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680473046 (103h22m15.277244178s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680982200 (103h22m15.277753227s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681478081 (103h22m15.27824911s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681895554 (103h22m15.278666585s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682340386 (103h22m15.279111418s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682758716 (103h22m15.279529749s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.683342542 (103h22m15.280113571s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:18.102504696 (103h22m15.699275711s) [upload=115702]: Next returned nil segment
Mar 10 18:22:18.102520582 (103h22m15.699291606s) [upload=115702]: Waiting for error group managing segments...
Mar 10 18:22:18.286488354 (103h22m15.883259367s) [noise=true , address=185.167.204.206:4037 , piece_id=TEYUSWWSWXEKS2TUKAXH7NXD57TQKEZLPS5IOYQVC54ZTY2LEHGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.291859985 (103h22m15.888631012s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.297964517 (103h22m15.894735561s) [noise=true , address=89.149.201.147:4005 , piece_id=CFTY5WE3SZLKOPH6KT4ZD3M2A7ZCIOH7BCUUPGTYM6NRFNJC2BZA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.307699767 (103h22m15.904470781s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.312994778 (103h22m15.909765791s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.319374438 (103h22m15.916145452s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.341507663 (103h22m15.938278696s) [noise=true , address=185.167.204.206:4054 , piece_id=GCJ4ZXR6YMUGI5BG2XOYP4FKTTTSPGVZ5UYS4BOMALGHCSBFQO7Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.351324302 (103h22m15.948095328s) [noise=true , address=81.171.14.70:4032 , piece_id=GPOUM37IH7EL7YANALUZPAQOQ2MHAFOZSVWM7VLDPCNRODOKOO5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.354375553 (103h22m15.951146587s) [noise=true , address=185.167.204.205:4032 , piece_id=YA3TR364FWAMYKQUROV4PYQWTGIXM7SLAUM3BN2FRVBW3MVILHDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.355210000 (103h22m15.951981024s) [noise=true , address=89.149.201.147:4020 , piece_id=AGOZMT6EYQKSFRYOJW4QAV2NMSPM4Q5LAI2SA276EKYK4X5ZGP5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.364878440 (103h22m15.961649464s) [noise=true , address=185.167.204.206:4039 , piece_id=4LSO5WGH5LQ5NIDHOHOACKVAYY37JEZHAUX66MEJ4FIGZ5LWDRFA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.366270355 (103h22m15.963041377s) [noise=true , address=185.167.204.206:4049 , piece_id=JPGUJCUADQNZU5KJAHY4FXUTKJVSEEGMZQ7HNUEWOIWYBJ3LYPEQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.383218910 (103h22m15.979989919s) [noise=true , address=185.167.204.205:4025 , piece_id=2BA453RCULDYAHVTMMVFTPT5MRC4VJNBKK3IHUOLSOMERCHA6LKA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.385734313 (103h22m15.982505326s) [noise=true , address=185.167.204.206:4062 , piece_id=LOWNTJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.386456163 (103h22m15.983227177s) [noise=true , address=185.167.204.206:4071 , piece_id=XT7Y2BSAL4MSA4P6GXDWGQ3R7TGBZT4ON7KAJNVUMBHYPPYU4YTA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389060733 (103h22m15.985831747s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389075564 (103h22m15.985846595s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389787069 (103h22m15.986558092s) [noise=true , address=185.167.204.205:4013 , piece_id=ZYGKKFJQRQZZVEO7DQ67E3ILC66TO5EQBXBAFMKRS2CMKKOKLLVA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.392844303 (103h22m15.989615315s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.395040677 (103h22m15.99181169s) [noise=true , address=185.167.204.206:4053 , piece_id=D72PWJW544HPN6C4C6YIIQBUP7ZIWIN4VCTSYRLGUSCIEUHBWMQA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397224921 (103h22m15.993995937s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397305588 (103h22m15.994076606s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.423243063 (103h22m16.020014089s) [noise=true , address=185.167.204.206:4067 , piece_id=XWH6ZTD6S3TN6TIFN5S4ZIBNR44VOYCB7UCKXOMBBM2MPMNSXUGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:24.985281962 (103h22m22.582052989s) [noise=true , address=185.167.204.205:4007 , piece_id=GDYOPUY5PJFCYLU7OYPBVTW424GT7FP77TMRYKYNKUWXZS6ENAYQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:30.233899651 (103h22m27.830670686s) [noise=true , address=185.167.204.205:4020 , piece_id=5GIMRHZGF2ZEFWLBL4U6KF4DKZ53JZ7PNWB4XTEQW5XCESHNZY5A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:23:17.302478912 (103h23m14.899249945s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection
Mar 10 18:23:17.302510476 (103h23m14.899281492s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 159.223.143.39:39206->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:23:17.302719428 (103h23m14.89949046s) [noise=true , address=185.167.204.206:4058 , piece_id=LZ6PUPONPGDVBLY3IIUUFUZENZTSTWNMCBEK4GVI3Z2RQXICZSDQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: expected piece hash; context deadline exceeded; EOF
Mar 10 18:23:17.302761551 (103h23m14.899532565s) [seg_pos={0 0}, upload=115702]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:23:17.401134721 (103h23m14.997905738s) [upload=115702]: Done uploading segments...
Mar 10 18:23:17.401207170 (103h23m14.997978196s) [upload=115702]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.564076528s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
644.732475ms trace:
https://jaeger.dev.storj.io/trace/62ea03c3a72aaf43
FAIL (2026-03-10 18:37:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 644.837494ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [................................................X...........] duration:
790.426057ms trace:
https://jaeger.dev.storj.io/trace/33e9eb6cbded500c
FAIL (2026-03-10 18:36:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 790.464667ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.........................................X..X...............] duration:
2.548874234s trace:
https://jaeger.dev.storj.io/trace/62b857fbddd8c134
FAIL (2026-03-10 18:36:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
TJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.678723726 (103h22m15.275494752s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679171303 (103h22m15.275942329s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679573269 (103h22m15.276344303s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680028932 (103h22m15.27679996s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680473046 (103h22m15.277244178s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680982200 (103h22m15.277753227s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681478081 (103h22m15.27824911s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681895554 (103h22m15.278666585s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682340386 (103h22m15.279111418s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682758716 (103h22m15.279529749s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.683342542 (103h22m15.280113571s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:18.102504696 (103h22m15.699275711s) [upload=115702]: Next returned nil segment
Mar 10 18:22:18.102520582 (103h22m15.699291606s) [upload=115702]: Waiting for error group managing segments...
Mar 10 18:22:18.286488354 (103h22m15.883259367s) [noise=true , address=185.167.204.206:4037 , piece_id=TEYUSWWSWXEKS2TUKAXH7NXD57TQKEZLPS5IOYQVC54ZTY2LEHGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.291859985 (103h22m15.888631012s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.297964517 (103h22m15.894735561s) [noise=true , address=89.149.201.147:4005 , piece_id=CFTY5WE3SZLKOPH6KT4ZD3M2A7ZCIOH7BCUUPGTYM6NRFNJC2BZA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.307699767 (103h22m15.904470781s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.312994778 (103h22m15.909765791s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.319374438 (103h22m15.916145452s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.341507663 (103h22m15.938278696s) [noise=true , address=185.167.204.206:4054 , piece_id=GCJ4ZXR6YMUGI5BG2XOYP4FKTTTSPGVZ5UYS4BOMALGHCSBFQO7Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.351324302 (103h22m15.948095328s) [noise=true , address=81.171.14.70:4032 , piece_id=GPOUM37IH7EL7YANALUZPAQOQ2MHAFOZSVWM7VLDPCNRODOKOO5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.354375553 (103h22m15.951146587s) [noise=true , address=185.167.204.205:4032 , piece_id=YA3TR364FWAMYKQUROV4PYQWTGIXM7SLAUM3BN2FRVBW3MVILHDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.355210000 (103h22m15.951981024s) [noise=true , address=89.149.201.147:4020 , piece_id=AGOZMT6EYQKSFRYOJW4QAV2NMSPM4Q5LAI2SA276EKYK4X5ZGP5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.364878440 (103h22m15.961649464s) [noise=true , address=185.167.204.206:4039 , piece_id=4LSO5WGH5LQ5NIDHOHOACKVAYY37JEZHAUX66MEJ4FIGZ5LWDRFA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.366270355 (103h22m15.963041377s) [noise=true , address=185.167.204.206:4049 , piece_id=JPGUJCUADQNZU5KJAHY4FXUTKJVSEEGMZQ7HNUEWOIWYBJ3LYPEQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.383218910 (103h22m15.979989919s) [noise=true , address=185.167.204.205:4025 , piece_id=2BA453RCULDYAHVTMMVFTPT5MRC4VJNBKK3IHUOLSOMERCHA6LKA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.385734313 (103h22m15.982505326s) [noise=true , address=185.167.204.206:4062 , piece_id=LOWNTJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.386456163 (103h22m15.983227177s) [noise=true , address=185.167.204.206:4071 , piece_id=XT7Y2BSAL4MSA4P6GXDWGQ3R7TGBZT4ON7KAJNVUMBHYPPYU4YTA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389060733 (103h22m15.985831747s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389075564 (103h22m15.985846595s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389787069 (103h22m15.986558092s) [noise=true , address=185.167.204.205:4013 , piece_id=ZYGKKFJQRQZZVEO7DQ67E3ILC66TO5EQBXBAFMKRS2CMKKOKLLVA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.392844303 (103h22m15.989615315s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.395040677 (103h22m15.99181169s) [noise=true , address=185.167.204.206:4053 , piece_id=D72PWJW544HPN6C4C6YIIQBUP7ZIWIN4VCTSYRLGUSCIEUHBWMQA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397224921 (103h22m15.993995937s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397305588 (103h22m15.994076606s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.423243063 (103h22m16.020014089s) [noise=true , address=185.167.204.206:4067 , piece_id=XWH6ZTD6S3TN6TIFN5S4ZIBNR44VOYCB7UCKXOMBBM2MPMNSXUGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:24.985281962 (103h22m22.582052989s) [noise=true , address=185.167.204.205:4007 , piece_id=GDYOPUY5PJFCYLU7OYPBVTW424GT7FP77TMRYKYNKUWXZS6ENAYQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:30.233899651 (103h22m27.830670686s) [noise=true , address=185.167.204.205:4020 , piece_id=5GIMRHZGF2ZEFWLBL4U6KF4DKZ53JZ7PNWB4XTEQW5XCESHNZY5A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:23:17.302478912 (103h23m14.899249945s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection
Mar 10 18:23:17.302510476 (103h23m14.899281492s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 159.223.143.39:39206->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:23:17.302719428 (103h23m14.89949046s) [noise=true , address=185.167.204.206:4058 , piece_id=LZ6PUPONPGDVBLY3IIUUFUZENZTSTWNMCBEK4GVI3Z2RQXICZSDQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: expected piece hash; context deadline exceeded; EOF
Mar 10 18:23:17.302761551 (103h23m14.899532565s) [seg_pos={0 0}, upload=115702]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:23:17.401134721 (103h23m14.997905738s) [upload=115702]: Done uploading segments...
Mar 10 18:23:17.401207170 (103h23m14.997978196s) [upload=115702]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.548981865s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
691.217152ms trace:
https://jaeger.dev.storj.io/trace/62ea03c3a72aaf43
FAIL (2026-03-10 18:36:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 691.251739ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
1.196520242s trace:
https://jaeger.dev.storj.io/trace/14d851485ec55b94
FAIL (2026-03-10 18:36:20 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
egments...
Mar 10 18:34:13.234105804 (103h34m10.830876822s) [upload=115928]: Got next segment. Inline: false
Mar 10 18:34:13.234171800 (103h34m10.830942823s) [upload=115928]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 10 18:34:13.234177690 (103h34m10.830948712s) [upload=115928]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 10 18:34:13.264565602 (103h34m10.861336626s) [seg_pos={0 0}, upload=115928]: Begin upload segment...
Mar 10 18:34:13.264650554 (103h34m10.861421568s) [seg_pos={0 0}, upload=115928]: Done begin upload segment.
Mar 10 18:34:13.264710110 (103h34m10.861481134s) [noise=true , address=45.91.116.9:4280 , piece_id=O6XIUDM5QEPO2S3TSGBRWFJOA6YNOPISRIKOKJMYIYVCY3W3O6FA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.265998754 (103h34m10.862769779s) [noise=true , address=77.90.31.14:4669 , piece_id=P7P7VFJ2CX2EXILR5AVF6ZQJSRWJ7HFBWB6APJUKQOETOPO7N3CQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.266563685 (103h34m10.863334713s) [noise=true , address=77.90.31.19:4863 , piece_id=DWBZ64GZUGWORUXIZGZVHTWA3VAB4GILLMMXVU5KB2VBGRX7FN2Q, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.266980797 (103h34m10.863751825s) [noise=true , address=193.149.164.68:4044 , piece_id=HBSJYACUD45I5CQBQ3PJZOWN4MBTXJCTGHMKHOMTUG5YCDISGPOA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.267514292 (103h34m10.864285328s) [noise=true , address=79.127.205.235:4262 , piece_id=X6XAWZ64BAJP2EYT4PXLQHP6NTPCMRXSGHPTDF55V4Z3XXUZVO7A, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.267576292 (103h34m10.864347316s) [noise=true , address=193.149.164.69:4071 , piece_id=WVYSHF5Z35Z4DFCEVWVBKRRDLELXAV5URWMSFHNKCYCBQBKXBXIA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.267944808 (103h34m10.864715836s) [noise=true , address=193.149.164.67:4032 , piece_id=OYVBK5VQX2ICUTG6LTGB5DOEUKWA2Q42ANILPBUA3ELWA5XZ7OSQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.268291059 (103h34m10.865062093s) [noise=true , address=79.127.205.234:4190 , piece_id=2DCWY6MVVYBU7R5R2G6LABDDL6YCBCYEQI3B5GL2P67SFCFU7ECQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.268635689 (103h34m10.865406733s) [noise=true , address=79.127.205.237:4241 , piece_id=EMX5GFPZX3VLRD4AG3SDATNVA3YSJ6EAB4WXY3LTFE6AWEO7XZ4A, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.268857349 (103h34m10.865628383s) [noise=true , address=77.90.31.11:4587 , piece_id=XDEV3CN5NW3VMYC7B3HYF2YLSQ6GV334LG2YYD2X4PIJQE3ZNEYA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.269379163 (103h34m10.866150189s) [noise=true , address=145.79.227.31:4822 , piece_id=7A2BFMPYL6EWUQC5RFZY6IIVXIVQLML6TSZVUP7VU4NIQLWYB63Q, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.269442067 (103h34m10.866213117s) [noise=true , address=77.90.31.13:4647 , piece_id=5DBAZJVJSJZF2AHFPAQBVTDKTF4LRXTJRNNLMSNDHD6G5TY366GQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.270023291 (103h34m10.866794323s) [noise=true , address=193.149.164.71:4298 , piece_id=VMRPXXIJIKZAWY6SCIA3FMAFIQ4XYI4LWXLVFRNMSKY5B6MV5DMQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.270558766 (103h34m10.867329801s) [noise=true , address=79.127.219.41:4954 , piece_id=UMD2OKNI36MLUAC5TOF6PVK573GN3JJSM3HXCYOLG25EGRPDVHJA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.271066172 (103h34m10.867837205s) [noise=true , address=77.90.31.18:4837 , piece_id=N5RMIIIGCAQES6G6YXHCUX2YRVKIE764F57XQFGWOCKYOBJV33ZA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.271143045 (103h34m10.867914074s) [noise=true , address=167.17.72.68:4645 , piece_id=76HO76ZYEW7YE2XGOZY3EJJHTZFW24B7IFGF3IMUV6IMYKTHLG2A, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.271561296 (103h34m10.86833233s) [noise=true , address=167.17.72.70:4755 , piece_id=NXGZWBH2UNVNSE7B2BHVBEC434N7MWHDYTDECVTZNB3O6GYPZUNA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.272067991 (103h34m10.868839022s) [noise=true , address=79.127.219.45:4878 , piece_id=GTM5BKXU6TQKMBCF5FOW34OKWVQLUJUJ4WINDYBZFR3ORI2OSHSA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.341870072 (103h34m10.938641097s) [upload=115928]: Next returned nil segment
Mar 10 18:34:13.341887354 (103h34m10.938658381s) [upload=115928]: Waiting for error group managing segments...
Mar 10 18:34:13.357317608 (103h34m10.954088641s) [noise=true , address=79.127.219.41:4954 , piece_id=UMD2OKNI36MLUAC5TOF6PVK573GN3JJSM3HXCYOLG25EGRPDVHJA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.369079638 (103h34m10.965850664s) [noise=true , address=45.91.116.9:4280 , piece_id=O6XIUDM5QEPO2S3TSGBRWFJOA6YNOPISRIKOKJMYIYVCY3W3O6FA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.383782380 (103h34m10.980553407s) [noise=true , address=145.79.227.31:4822 , piece_id=7A2BFMPYL6EWUQC5RFZY6IIVXIVQLML6TSZVUP7VU4NIQLWYB63Q, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.383844707 (103h34m10.980615732s) [noise=true , address=79.127.219.45:4878 , piece_id=GTM5BKXU6TQKMBCF5FOW34OKWVQLUJUJ4WINDYBZFR3ORI2OSHSA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.596292826 (103h34m11.19306384s) [noise=true , address=167.17.72.68:4645 , piece_id=76HO76ZYEW7YE2XGOZY3EJJHTZFW24B7IFGF3IMUV6IMYKTHLG2A, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.657877077 (103h34m11.254648103s) [noise=true , address=167.17.72.70:4755 , piece_id=NXGZWBH2UNVNSE7B2BHVBEC434N7MWHDYTDECVTZNB3O6GYPZUNA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.815668472 (103h34m11.412439501s) [noise=true , address=79.127.205.234:4190 , piece_id=2DCWY6MVVYBU7R5R2G6LABDDL6YCBCYEQI3B5GL2P67SFCFU7ECQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.816829420 (103h34m11.413600446s) [noise=true , address=79.127.205.237:4241 , piece_id=EMX5GFPZX3VLRD4AG3SDATNVA3YSJ6EAB4WXY3LTFE6AWEO7XZ4A, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.818063674 (103h34m11.414834697s) [noise=true , address=79.127.205.235:4262 , piece_id=X6XAWZ64BAJP2EYT4PXLQHP6NTPCMRXSGHPTDF55V4Z3XXUZVO7A, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.844922561 (103h34m11.441693585s) [noise=true , address=193.149.164.67:4032 , piece_id=OYVBK5VQX2ICUTG6LTGB5DOEUKWA2Q42ANILPBUA3ELWA5XZ7OSQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.862417635 (103h34m11.459188668s) [noise=true , address=193.149.164.71:4298 , piece_id=VMRPXXIJIKZAWY6SCIA3FMAFIQ4XYI4LWXLVFRNMSKY5B6MV5DMQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.864932308 (103h34m11.461703323s) [noise=true , address=193.149.164.69:4071 , piece_id=WVYSHF5Z35Z4DFCEVWVBKRRDLELXAV5URWMSFHNKCYCBQBKXBXIA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.874686150 (103h34m11.471457173s) [noise=true , address=193.149.164.68:4044 , piece_id=HBSJYACUD45I5CQBQ3PJZOWN4MBTXJCTGHMKHOMTUG5YCDISGPOA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:35:13.203204147 (103h35m10.799975179s) [noise=true , address=77.90.31.19:4863 , piece_id=DWBZ64GZUGWORUXIZGZVHTWA3VAB4GILLMMXVU5KB2VBGRX7FN2Q, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL, address:77.90.31.19:4863): protocol: rpc: dial tcp 77.90.31.19:4863: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.203494517 (103h35m10.800265546s) [noise=true , address=77.90.31.18:4837 , piece_id=N5RMIIIGCAQES6G6YXHCUX2YRVKIE764F57XQFGWOCKYOBJV33ZA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:12YfSnynVtsWn1XtXHGeik1vUgqzRtArmEJJrWcXd23m2yUTS8j, address:77.90.31.18:4837): protocol: write tcp 159.223.143.39:56048->77.90.31.18:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.203740816 (103h35m10.800511848s) [noise=true , address=77.90.31.13:4647 , piece_id=5DBAZJVJSJZF2AHFPAQBVTDKTF4LRXTJRNNLMSNDHD6G5TY366GQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:121XgCUbGhKnfnTDNasmpgsQzzkZAqnFie53A7pB793wKT9VCZv, address:77.90.31.13:4647): protocol: rpc: dial tcp 77.90.31.13:4647: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.203754450 (103h35m10.800525479s) [noise=true , address=77.90.31.11:4587 , piece_id=XDEV3CN5NW3VMYC7B3HYF2YLSQ6GV334LG2YYD2X4PIJQE3ZNEYA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:1twkDDcpRrmR5bFhjGKKccMxC4Y35xh7tLaFxawhBmZkVWh3Bb, address:77.90.31.11:4587): protocol: rpc: dial tcp 77.90.31.11:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.204001069 (103h35m10.800772099s) [noise=true , address=77.90.31.14:4669 , piece_id=P7P7VFJ2CX2EXILR5AVF6ZQJSRWJ7HFBWB6APJUKQOETOPO7N3CQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.204030719 (103h35m10.800801732s) [seg_pos={0 0}, upload=115928]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Mar 10 18:35:13.239310552 (103h35m10.836081578s) [upload=115928]: Done uploading segments...
Mar 10 18:35:13.239351234 (103h35m10.836122261s) [upload=115928]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 1.196618118s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................................................X..........] duration:
722.637477ms trace:
https://jaeger.dev.storj.io/trace/33e9eb6cbded500c
FAIL (2026-03-10 18:36:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 722.687014ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
732.248173ms trace:
https://jaeger.dev.storj.io/trace/62ea03c3a72aaf43
FAIL (2026-03-10 18:35:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 732.300711ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..........................................X..X..............] duration:
2.543143327s trace:
https://jaeger.dev.storj.io/trace/62b857fbddd8c134
FAIL (2026-03-10 18:35:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
TJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.678723726 (103h22m15.275494752s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679171303 (103h22m15.275942329s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679573269 (103h22m15.276344303s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680028932 (103h22m15.27679996s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680473046 (103h22m15.277244178s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680982200 (103h22m15.277753227s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681478081 (103h22m15.27824911s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681895554 (103h22m15.278666585s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682340386 (103h22m15.279111418s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682758716 (103h22m15.279529749s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.683342542 (103h22m15.280113571s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:18.102504696 (103h22m15.699275711s) [upload=115702]: Next returned nil segment
Mar 10 18:22:18.102520582 (103h22m15.699291606s) [upload=115702]: Waiting for error group managing segments...
Mar 10 18:22:18.286488354 (103h22m15.883259367s) [noise=true , address=185.167.204.206:4037 , piece_id=TEYUSWWSWXEKS2TUKAXH7NXD57TQKEZLPS5IOYQVC54ZTY2LEHGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.291859985 (103h22m15.888631012s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.297964517 (103h22m15.894735561s) [noise=true , address=89.149.201.147:4005 , piece_id=CFTY5WE3SZLKOPH6KT4ZD3M2A7ZCIOH7BCUUPGTYM6NRFNJC2BZA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.307699767 (103h22m15.904470781s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.312994778 (103h22m15.909765791s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.319374438 (103h22m15.916145452s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.341507663 (103h22m15.938278696s) [noise=true , address=185.167.204.206:4054 , piece_id=GCJ4ZXR6YMUGI5BG2XOYP4FKTTTSPGVZ5UYS4BOMALGHCSBFQO7Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.351324302 (103h22m15.948095328s) [noise=true , address=81.171.14.70:4032 , piece_id=GPOUM37IH7EL7YANALUZPAQOQ2MHAFOZSVWM7VLDPCNRODOKOO5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.354375553 (103h22m15.951146587s) [noise=true , address=185.167.204.205:4032 , piece_id=YA3TR364FWAMYKQUROV4PYQWTGIXM7SLAUM3BN2FRVBW3MVILHDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.355210000 (103h22m15.951981024s) [noise=true , address=89.149.201.147:4020 , piece_id=AGOZMT6EYQKSFRYOJW4QAV2NMSPM4Q5LAI2SA276EKYK4X5ZGP5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.364878440 (103h22m15.961649464s) [noise=true , address=185.167.204.206:4039 , piece_id=4LSO5WGH5LQ5NIDHOHOACKVAYY37JEZHAUX66MEJ4FIGZ5LWDRFA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.366270355 (103h22m15.963041377s) [noise=true , address=185.167.204.206:4049 , piece_id=JPGUJCUADQNZU5KJAHY4FXUTKJVSEEGMZQ7HNUEWOIWYBJ3LYPEQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.383218910 (103h22m15.979989919s) [noise=true , address=185.167.204.205:4025 , piece_id=2BA453RCULDYAHVTMMVFTPT5MRC4VJNBKK3IHUOLSOMERCHA6LKA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.385734313 (103h22m15.982505326s) [noise=true , address=185.167.204.206:4062 , piece_id=LOWNTJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.386456163 (103h22m15.983227177s) [noise=true , address=185.167.204.206:4071 , piece_id=XT7Y2BSAL4MSA4P6GXDWGQ3R7TGBZT4ON7KAJNVUMBHYPPYU4YTA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389060733 (103h22m15.985831747s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389075564 (103h22m15.985846595s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389787069 (103h22m15.986558092s) [noise=true , address=185.167.204.205:4013 , piece_id=ZYGKKFJQRQZZVEO7DQ67E3ILC66TO5EQBXBAFMKRS2CMKKOKLLVA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.392844303 (103h22m15.989615315s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.395040677 (103h22m15.99181169s) [noise=true , address=185.167.204.206:4053 , piece_id=D72PWJW544HPN6C4C6YIIQBUP7ZIWIN4VCTSYRLGUSCIEUHBWMQA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397224921 (103h22m15.993995937s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397305588 (103h22m15.994076606s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.423243063 (103h22m16.020014089s) [noise=true , address=185.167.204.206:4067 , piece_id=XWH6ZTD6S3TN6TIFN5S4ZIBNR44VOYCB7UCKXOMBBM2MPMNSXUGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:24.985281962 (103h22m22.582052989s) [noise=true , address=185.167.204.205:4007 , piece_id=GDYOPUY5PJFCYLU7OYPBVTW424GT7FP77TMRYKYNKUWXZS6ENAYQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:30.233899651 (103h22m27.830670686s) [noise=true , address=185.167.204.205:4020 , piece_id=5GIMRHZGF2ZEFWLBL4U6KF4DKZ53JZ7PNWB4XTEQW5XCESHNZY5A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:23:17.302478912 (103h23m14.899249945s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection
Mar 10 18:23:17.302510476 (103h23m14.899281492s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 159.223.143.39:39206->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:23:17.302719428 (103h23m14.89949046s) [noise=true , address=185.167.204.206:4058 , piece_id=LZ6PUPONPGDVBLY3IIUUFUZENZTSTWNMCBEK4GVI3Z2RQXICZSDQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: expected piece hash; context deadline exceeded; EOF
Mar 10 18:23:17.302761551 (103h23m14.899532565s) [seg_pos={0 0}, upload=115702]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:23:17.401134721 (103h23m14.997905738s) [upload=115702]: Done uploading segments...
Mar 10 18:23:17.401207170 (103h23m14.997978196s) [upload=115702]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.543172063s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
918.439369ms trace:
https://jaeger.dev.storj.io/trace/62ea03c3a72aaf43
FAIL (2026-03-10 18:35:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 918.472081ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
1.233589823s trace:
https://jaeger.dev.storj.io/trace/14d851485ec55b94
FAIL (2026-03-10 18:35:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
egments...
Mar 10 18:34:13.234105804 (103h34m10.830876822s) [upload=115928]: Got next segment. Inline: false
Mar 10 18:34:13.234171800 (103h34m10.830942823s) [upload=115928]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 10 18:34:13.234177690 (103h34m10.830948712s) [upload=115928]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 10 18:34:13.264565602 (103h34m10.861336626s) [seg_pos={0 0}, upload=115928]: Begin upload segment...
Mar 10 18:34:13.264650554 (103h34m10.861421568s) [seg_pos={0 0}, upload=115928]: Done begin upload segment.
Mar 10 18:34:13.264710110 (103h34m10.861481134s) [noise=true , address=45.91.116.9:4280 , piece_id=O6XIUDM5QEPO2S3TSGBRWFJOA6YNOPISRIKOKJMYIYVCY3W3O6FA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.265998754 (103h34m10.862769779s) [noise=true , address=77.90.31.14:4669 , piece_id=P7P7VFJ2CX2EXILR5AVF6ZQJSRWJ7HFBWB6APJUKQOETOPO7N3CQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.266563685 (103h34m10.863334713s) [noise=true , address=77.90.31.19:4863 , piece_id=DWBZ64GZUGWORUXIZGZVHTWA3VAB4GILLMMXVU5KB2VBGRX7FN2Q, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.266980797 (103h34m10.863751825s) [noise=true , address=193.149.164.68:4044 , piece_id=HBSJYACUD45I5CQBQ3PJZOWN4MBTXJCTGHMKHOMTUG5YCDISGPOA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.267514292 (103h34m10.864285328s) [noise=true , address=79.127.205.235:4262 , piece_id=X6XAWZ64BAJP2EYT4PXLQHP6NTPCMRXSGHPTDF55V4Z3XXUZVO7A, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.267576292 (103h34m10.864347316s) [noise=true , address=193.149.164.69:4071 , piece_id=WVYSHF5Z35Z4DFCEVWVBKRRDLELXAV5URWMSFHNKCYCBQBKXBXIA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.267944808 (103h34m10.864715836s) [noise=true , address=193.149.164.67:4032 , piece_id=OYVBK5VQX2ICUTG6LTGB5DOEUKWA2Q42ANILPBUA3ELWA5XZ7OSQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.268291059 (103h34m10.865062093s) [noise=true , address=79.127.205.234:4190 , piece_id=2DCWY6MVVYBU7R5R2G6LABDDL6YCBCYEQI3B5GL2P67SFCFU7ECQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.268635689 (103h34m10.865406733s) [noise=true , address=79.127.205.237:4241 , piece_id=EMX5GFPZX3VLRD4AG3SDATNVA3YSJ6EAB4WXY3LTFE6AWEO7XZ4A, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.268857349 (103h34m10.865628383s) [noise=true , address=77.90.31.11:4587 , piece_id=XDEV3CN5NW3VMYC7B3HYF2YLSQ6GV334LG2YYD2X4PIJQE3ZNEYA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.269379163 (103h34m10.866150189s) [noise=true , address=145.79.227.31:4822 , piece_id=7A2BFMPYL6EWUQC5RFZY6IIVXIVQLML6TSZVUP7VU4NIQLWYB63Q, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.269442067 (103h34m10.866213117s) [noise=true , address=77.90.31.13:4647 , piece_id=5DBAZJVJSJZF2AHFPAQBVTDKTF4LRXTJRNNLMSNDHD6G5TY366GQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.270023291 (103h34m10.866794323s) [noise=true , address=193.149.164.71:4298 , piece_id=VMRPXXIJIKZAWY6SCIA3FMAFIQ4XYI4LWXLVFRNMSKY5B6MV5DMQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.270558766 (103h34m10.867329801s) [noise=true , address=79.127.219.41:4954 , piece_id=UMD2OKNI36MLUAC5TOF6PVK573GN3JJSM3HXCYOLG25EGRPDVHJA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.271066172 (103h34m10.867837205s) [noise=true , address=77.90.31.18:4837 , piece_id=N5RMIIIGCAQES6G6YXHCUX2YRVKIE764F57XQFGWOCKYOBJV33ZA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.271143045 (103h34m10.867914074s) [noise=true , address=167.17.72.68:4645 , piece_id=76HO76ZYEW7YE2XGOZY3EJJHTZFW24B7IFGF3IMUV6IMYKTHLG2A, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.271561296 (103h34m10.86833233s) [noise=true , address=167.17.72.70:4755 , piece_id=NXGZWBH2UNVNSE7B2BHVBEC434N7MWHDYTDECVTZNB3O6GYPZUNA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.272067991 (103h34m10.868839022s) [noise=true , address=79.127.219.45:4878 , piece_id=GTM5BKXU6TQKMBCF5FOW34OKWVQLUJUJ4WINDYBZFR3ORI2OSHSA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.341870072 (103h34m10.938641097s) [upload=115928]: Next returned nil segment
Mar 10 18:34:13.341887354 (103h34m10.938658381s) [upload=115928]: Waiting for error group managing segments...
Mar 10 18:34:13.357317608 (103h34m10.954088641s) [noise=true , address=79.127.219.41:4954 , piece_id=UMD2OKNI36MLUAC5TOF6PVK573GN3JJSM3HXCYOLG25EGRPDVHJA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.369079638 (103h34m10.965850664s) [noise=true , address=45.91.116.9:4280 , piece_id=O6XIUDM5QEPO2S3TSGBRWFJOA6YNOPISRIKOKJMYIYVCY3W3O6FA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.383782380 (103h34m10.980553407s) [noise=true , address=145.79.227.31:4822 , piece_id=7A2BFMPYL6EWUQC5RFZY6IIVXIVQLML6TSZVUP7VU4NIQLWYB63Q, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.383844707 (103h34m10.980615732s) [noise=true , address=79.127.219.45:4878 , piece_id=GTM5BKXU6TQKMBCF5FOW34OKWVQLUJUJ4WINDYBZFR3ORI2OSHSA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.596292826 (103h34m11.19306384s) [noise=true , address=167.17.72.68:4645 , piece_id=76HO76ZYEW7YE2XGOZY3EJJHTZFW24B7IFGF3IMUV6IMYKTHLG2A, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.657877077 (103h34m11.254648103s) [noise=true , address=167.17.72.70:4755 , piece_id=NXGZWBH2UNVNSE7B2BHVBEC434N7MWHDYTDECVTZNB3O6GYPZUNA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.815668472 (103h34m11.412439501s) [noise=true , address=79.127.205.234:4190 , piece_id=2DCWY6MVVYBU7R5R2G6LABDDL6YCBCYEQI3B5GL2P67SFCFU7ECQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.816829420 (103h34m11.413600446s) [noise=true , address=79.127.205.237:4241 , piece_id=EMX5GFPZX3VLRD4AG3SDATNVA3YSJ6EAB4WXY3LTFE6AWEO7XZ4A, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.818063674 (103h34m11.414834697s) [noise=true , address=79.127.205.235:4262 , piece_id=X6XAWZ64BAJP2EYT4PXLQHP6NTPCMRXSGHPTDF55V4Z3XXUZVO7A, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.844922561 (103h34m11.441693585s) [noise=true , address=193.149.164.67:4032 , piece_id=OYVBK5VQX2ICUTG6LTGB5DOEUKWA2Q42ANILPBUA3ELWA5XZ7OSQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.862417635 (103h34m11.459188668s) [noise=true , address=193.149.164.71:4298 , piece_id=VMRPXXIJIKZAWY6SCIA3FMAFIQ4XYI4LWXLVFRNMSKY5B6MV5DMQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.864932308 (103h34m11.461703323s) [noise=true , address=193.149.164.69:4071 , piece_id=WVYSHF5Z35Z4DFCEVWVBKRRDLELXAV5URWMSFHNKCYCBQBKXBXIA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.874686150 (103h34m11.471457173s) [noise=true , address=193.149.164.68:4044 , piece_id=HBSJYACUD45I5CQBQ3PJZOWN4MBTXJCTGHMKHOMTUG5YCDISGPOA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:35:13.203204147 (103h35m10.799975179s) [noise=true , address=77.90.31.19:4863 , piece_id=DWBZ64GZUGWORUXIZGZVHTWA3VAB4GILLMMXVU5KB2VBGRX7FN2Q, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL, address:77.90.31.19:4863): protocol: rpc: dial tcp 77.90.31.19:4863: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.203494517 (103h35m10.800265546s) [noise=true , address=77.90.31.18:4837 , piece_id=N5RMIIIGCAQES6G6YXHCUX2YRVKIE764F57XQFGWOCKYOBJV33ZA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:12YfSnynVtsWn1XtXHGeik1vUgqzRtArmEJJrWcXd23m2yUTS8j, address:77.90.31.18:4837): protocol: write tcp 159.223.143.39:56048->77.90.31.18:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.203740816 (103h35m10.800511848s) [noise=true , address=77.90.31.13:4647 , piece_id=5DBAZJVJSJZF2AHFPAQBVTDKTF4LRXTJRNNLMSNDHD6G5TY366GQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:121XgCUbGhKnfnTDNasmpgsQzzkZAqnFie53A7pB793wKT9VCZv, address:77.90.31.13:4647): protocol: rpc: dial tcp 77.90.31.13:4647: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.203754450 (103h35m10.800525479s) [noise=true , address=77.90.31.11:4587 , piece_id=XDEV3CN5NW3VMYC7B3HYF2YLSQ6GV334LG2YYD2X4PIJQE3ZNEYA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:1twkDDcpRrmR5bFhjGKKccMxC4Y35xh7tLaFxawhBmZkVWh3Bb, address:77.90.31.11:4587): protocol: rpc: dial tcp 77.90.31.11:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.204001069 (103h35m10.800772099s) [noise=true , address=77.90.31.14:4669 , piece_id=P7P7VFJ2CX2EXILR5AVF6ZQJSRWJ7HFBWB6APJUKQOETOPO7N3CQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.204030719 (103h35m10.800801732s) [seg_pos={0 0}, upload=115928]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Mar 10 18:35:13.239310552 (103h35m10.836081578s) [upload=115928]: Done uploading segments...
Mar 10 18:35:13.239351234 (103h35m10.836122261s) [upload=115928]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 1.233611297s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.038498584s trace:
https://jaeger.dev.storj.io/trace/14d851485ec55b94
FAIL (2026-03-10 18:35:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
egments...
Mar 10 18:34:13.234105804 (103h34m10.830876822s) [upload=115928]: Got next segment. Inline: false
Mar 10 18:34:13.234171800 (103h34m10.830942823s) [upload=115928]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 10 18:34:13.234177690 (103h34m10.830948712s) [upload=115928]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 10 18:34:13.264565602 (103h34m10.861336626s) [seg_pos={0 0}, upload=115928]: Begin upload segment...
Mar 10 18:34:13.264650554 (103h34m10.861421568s) [seg_pos={0 0}, upload=115928]: Done begin upload segment.
Mar 10 18:34:13.264710110 (103h34m10.861481134s) [noise=true , address=45.91.116.9:4280 , piece_id=O6XIUDM5QEPO2S3TSGBRWFJOA6YNOPISRIKOKJMYIYVCY3W3O6FA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.265998754 (103h34m10.862769779s) [noise=true , address=77.90.31.14:4669 , piece_id=P7P7VFJ2CX2EXILR5AVF6ZQJSRWJ7HFBWB6APJUKQOETOPO7N3CQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.266563685 (103h34m10.863334713s) [noise=true , address=77.90.31.19:4863 , piece_id=DWBZ64GZUGWORUXIZGZVHTWA3VAB4GILLMMXVU5KB2VBGRX7FN2Q, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.266980797 (103h34m10.863751825s) [noise=true , address=193.149.164.68:4044 , piece_id=HBSJYACUD45I5CQBQ3PJZOWN4MBTXJCTGHMKHOMTUG5YCDISGPOA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.267514292 (103h34m10.864285328s) [noise=true , address=79.127.205.235:4262 , piece_id=X6XAWZ64BAJP2EYT4PXLQHP6NTPCMRXSGHPTDF55V4Z3XXUZVO7A, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.267576292 (103h34m10.864347316s) [noise=true , address=193.149.164.69:4071 , piece_id=WVYSHF5Z35Z4DFCEVWVBKRRDLELXAV5URWMSFHNKCYCBQBKXBXIA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.267944808 (103h34m10.864715836s) [noise=true , address=193.149.164.67:4032 , piece_id=OYVBK5VQX2ICUTG6LTGB5DOEUKWA2Q42ANILPBUA3ELWA5XZ7OSQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.268291059 (103h34m10.865062093s) [noise=true , address=79.127.205.234:4190 , piece_id=2DCWY6MVVYBU7R5R2G6LABDDL6YCBCYEQI3B5GL2P67SFCFU7ECQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.268635689 (103h34m10.865406733s) [noise=true , address=79.127.205.237:4241 , piece_id=EMX5GFPZX3VLRD4AG3SDATNVA3YSJ6EAB4WXY3LTFE6AWEO7XZ4A, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.268857349 (103h34m10.865628383s) [noise=true , address=77.90.31.11:4587 , piece_id=XDEV3CN5NW3VMYC7B3HYF2YLSQ6GV334LG2YYD2X4PIJQE3ZNEYA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.269379163 (103h34m10.866150189s) [noise=true , address=145.79.227.31:4822 , piece_id=7A2BFMPYL6EWUQC5RFZY6IIVXIVQLML6TSZVUP7VU4NIQLWYB63Q, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.269442067 (103h34m10.866213117s) [noise=true , address=77.90.31.13:4647 , piece_id=5DBAZJVJSJZF2AHFPAQBVTDKTF4LRXTJRNNLMSNDHD6G5TY366GQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.270023291 (103h34m10.866794323s) [noise=true , address=193.149.164.71:4298 , piece_id=VMRPXXIJIKZAWY6SCIA3FMAFIQ4XYI4LWXLVFRNMSKY5B6MV5DMQ, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.270558766 (103h34m10.867329801s) [noise=true , address=79.127.219.41:4954 , piece_id=UMD2OKNI36MLUAC5TOF6PVK573GN3JJSM3HXCYOLG25EGRPDVHJA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.271066172 (103h34m10.867837205s) [noise=true , address=77.90.31.18:4837 , piece_id=N5RMIIIGCAQES6G6YXHCUX2YRVKIE764F57XQFGWOCKYOBJV33ZA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.271143045 (103h34m10.867914074s) [noise=true , address=167.17.72.68:4645 , piece_id=76HO76ZYEW7YE2XGOZY3EJJHTZFW24B7IFGF3IMUV6IMYKTHLG2A, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.271561296 (103h34m10.86833233s) [noise=true , address=167.17.72.70:4755 , piece_id=NXGZWBH2UNVNSE7B2BHVBEC434N7MWHDYTDECVTZNB3O6GYPZUNA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.272067991 (103h34m10.868839022s) [noise=true , address=79.127.219.45:4878 , piece_id=GTM5BKXU6TQKMBCF5FOW34OKWVQLUJUJ4WINDYBZFR3ORI2OSHSA, seg_pos={0 0}, upload=115928]: Uploading piece...
Mar 10 18:34:13.341870072 (103h34m10.938641097s) [upload=115928]: Next returned nil segment
Mar 10 18:34:13.341887354 (103h34m10.938658381s) [upload=115928]: Waiting for error group managing segments...
Mar 10 18:34:13.357317608 (103h34m10.954088641s) [noise=true , address=79.127.219.41:4954 , piece_id=UMD2OKNI36MLUAC5TOF6PVK573GN3JJSM3HXCYOLG25EGRPDVHJA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.369079638 (103h34m10.965850664s) [noise=true , address=45.91.116.9:4280 , piece_id=O6XIUDM5QEPO2S3TSGBRWFJOA6YNOPISRIKOKJMYIYVCY3W3O6FA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.383782380 (103h34m10.980553407s) [noise=true , address=145.79.227.31:4822 , piece_id=7A2BFMPYL6EWUQC5RFZY6IIVXIVQLML6TSZVUP7VU4NIQLWYB63Q, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.383844707 (103h34m10.980615732s) [noise=true , address=79.127.219.45:4878 , piece_id=GTM5BKXU6TQKMBCF5FOW34OKWVQLUJUJ4WINDYBZFR3ORI2OSHSA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.596292826 (103h34m11.19306384s) [noise=true , address=167.17.72.68:4645 , piece_id=76HO76ZYEW7YE2XGOZY3EJJHTZFW24B7IFGF3IMUV6IMYKTHLG2A, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.657877077 (103h34m11.254648103s) [noise=true , address=167.17.72.70:4755 , piece_id=NXGZWBH2UNVNSE7B2BHVBEC434N7MWHDYTDECVTZNB3O6GYPZUNA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.815668472 (103h34m11.412439501s) [noise=true , address=79.127.205.234:4190 , piece_id=2DCWY6MVVYBU7R5R2G6LABDDL6YCBCYEQI3B5GL2P67SFCFU7ECQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.816829420 (103h34m11.413600446s) [noise=true , address=79.127.205.237:4241 , piece_id=EMX5GFPZX3VLRD4AG3SDATNVA3YSJ6EAB4WXY3LTFE6AWEO7XZ4A, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.818063674 (103h34m11.414834697s) [noise=true , address=79.127.205.235:4262 , piece_id=X6XAWZ64BAJP2EYT4PXLQHP6NTPCMRXSGHPTDF55V4Z3XXUZVO7A, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.844922561 (103h34m11.441693585s) [noise=true , address=193.149.164.67:4032 , piece_id=OYVBK5VQX2ICUTG6LTGB5DOEUKWA2Q42ANILPBUA3ELWA5XZ7OSQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.862417635 (103h34m11.459188668s) [noise=true , address=193.149.164.71:4298 , piece_id=VMRPXXIJIKZAWY6SCIA3FMAFIQ4XYI4LWXLVFRNMSKY5B6MV5DMQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.864932308 (103h34m11.461703323s) [noise=true , address=193.149.164.69:4071 , piece_id=WVYSHF5Z35Z4DFCEVWVBKRRDLELXAV5URWMSFHNKCYCBQBKXBXIA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:34:13.874686150 (103h34m11.471457173s) [noise=true , address=193.149.164.68:4044 , piece_id=HBSJYACUD45I5CQBQ3PJZOWN4MBTXJCTGHMKHOMTUG5YCDISGPOA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: <nil>
Mar 10 18:35:13.203204147 (103h35m10.799975179s) [noise=true , address=77.90.31.19:4863 , piece_id=DWBZ64GZUGWORUXIZGZVHTWA3VAB4GILLMMXVU5KB2VBGRX7FN2Q, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL, address:77.90.31.19:4863): protocol: rpc: dial tcp 77.90.31.19:4863: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.203494517 (103h35m10.800265546s) [noise=true , address=77.90.31.18:4837 , piece_id=N5RMIIIGCAQES6G6YXHCUX2YRVKIE764F57XQFGWOCKYOBJV33ZA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:12YfSnynVtsWn1XtXHGeik1vUgqzRtArmEJJrWcXd23m2yUTS8j, address:77.90.31.18:4837): protocol: write tcp 159.223.143.39:56048->77.90.31.18:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.203740816 (103h35m10.800511848s) [noise=true , address=77.90.31.13:4647 , piece_id=5DBAZJVJSJZF2AHFPAQBVTDKTF4LRXTJRNNLMSNDHD6G5TY366GQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:121XgCUbGhKnfnTDNasmpgsQzzkZAqnFie53A7pB793wKT9VCZv, address:77.90.31.13:4647): protocol: rpc: dial tcp 77.90.31.13:4647: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.203754450 (103h35m10.800525479s) [noise=true , address=77.90.31.11:4587 , piece_id=XDEV3CN5NW3VMYC7B3HYF2YLSQ6GV334LG2YYD2X4PIJQE3ZNEYA, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:1twkDDcpRrmR5bFhjGKKccMxC4Y35xh7tLaFxawhBmZkVWh3Bb, address:77.90.31.11:4587): protocol: rpc: dial tcp 77.90.31.11:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.204001069 (103h35m10.800772099s) [noise=true , address=77.90.31.14:4669 , piece_id=P7P7VFJ2CX2EXILR5AVF6ZQJSRWJ7HFBWB6APJUKQOETOPO7N3CQ, seg_pos={0 0}, upload=115928]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:35:13.204030719 (103h35m10.800801732s) [seg_pos={0 0}, upload=115928]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Mar 10 18:35:13.239310552 (103h35m10.836081578s) [upload=115928]: Done uploading segments...
Mar 10 18:35:13.239351234 (103h35m10.836122261s) [upload=115928]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 1m0.038530908s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..................................................X.........] duration:
513.98718ms trace:
https://jaeger.dev.storj.io/trace/33e9eb6cbded500c
FAIL (2026-03-10 18:35:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 514.052187ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...........................................X..X.............] duration:
2.273483402s trace:
https://jaeger.dev.storj.io/trace/62b857fbddd8c134
FAIL (2026-03-10 18:34:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
TJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.678723726 (103h22m15.275494752s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679171303 (103h22m15.275942329s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679573269 (103h22m15.276344303s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680028932 (103h22m15.27679996s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680473046 (103h22m15.277244178s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680982200 (103h22m15.277753227s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681478081 (103h22m15.27824911s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681895554 (103h22m15.278666585s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682340386 (103h22m15.279111418s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682758716 (103h22m15.279529749s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.683342542 (103h22m15.280113571s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:18.102504696 (103h22m15.699275711s) [upload=115702]: Next returned nil segment
Mar 10 18:22:18.102520582 (103h22m15.699291606s) [upload=115702]: Waiting for error group managing segments...
Mar 10 18:22:18.286488354 (103h22m15.883259367s) [noise=true , address=185.167.204.206:4037 , piece_id=TEYUSWWSWXEKS2TUKAXH7NXD57TQKEZLPS5IOYQVC54ZTY2LEHGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.291859985 (103h22m15.888631012s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.297964517 (103h22m15.894735561s) [noise=true , address=89.149.201.147:4005 , piece_id=CFTY5WE3SZLKOPH6KT4ZD3M2A7ZCIOH7BCUUPGTYM6NRFNJC2BZA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.307699767 (103h22m15.904470781s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.312994778 (103h22m15.909765791s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.319374438 (103h22m15.916145452s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.341507663 (103h22m15.938278696s) [noise=true , address=185.167.204.206:4054 , piece_id=GCJ4ZXR6YMUGI5BG2XOYP4FKTTTSPGVZ5UYS4BOMALGHCSBFQO7Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.351324302 (103h22m15.948095328s) [noise=true , address=81.171.14.70:4032 , piece_id=GPOUM37IH7EL7YANALUZPAQOQ2MHAFOZSVWM7VLDPCNRODOKOO5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.354375553 (103h22m15.951146587s) [noise=true , address=185.167.204.205:4032 , piece_id=YA3TR364FWAMYKQUROV4PYQWTGIXM7SLAUM3BN2FRVBW3MVILHDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.355210000 (103h22m15.951981024s) [noise=true , address=89.149.201.147:4020 , piece_id=AGOZMT6EYQKSFRYOJW4QAV2NMSPM4Q5LAI2SA276EKYK4X5ZGP5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.364878440 (103h22m15.961649464s) [noise=true , address=185.167.204.206:4039 , piece_id=4LSO5WGH5LQ5NIDHOHOACKVAYY37JEZHAUX66MEJ4FIGZ5LWDRFA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.366270355 (103h22m15.963041377s) [noise=true , address=185.167.204.206:4049 , piece_id=JPGUJCUADQNZU5KJAHY4FXUTKJVSEEGMZQ7HNUEWOIWYBJ3LYPEQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.383218910 (103h22m15.979989919s) [noise=true , address=185.167.204.205:4025 , piece_id=2BA453RCULDYAHVTMMVFTPT5MRC4VJNBKK3IHUOLSOMERCHA6LKA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.385734313 (103h22m15.982505326s) [noise=true , address=185.167.204.206:4062 , piece_id=LOWNTJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.386456163 (103h22m15.983227177s) [noise=true , address=185.167.204.206:4071 , piece_id=XT7Y2BSAL4MSA4P6GXDWGQ3R7TGBZT4ON7KAJNVUMBHYPPYU4YTA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389060733 (103h22m15.985831747s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389075564 (103h22m15.985846595s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389787069 (103h22m15.986558092s) [noise=true , address=185.167.204.205:4013 , piece_id=ZYGKKFJQRQZZVEO7DQ67E3ILC66TO5EQBXBAFMKRS2CMKKOKLLVA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.392844303 (103h22m15.989615315s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.395040677 (103h22m15.99181169s) [noise=true , address=185.167.204.206:4053 , piece_id=D72PWJW544HPN6C4C6YIIQBUP7ZIWIN4VCTSYRLGUSCIEUHBWMQA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397224921 (103h22m15.993995937s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397305588 (103h22m15.994076606s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.423243063 (103h22m16.020014089s) [noise=true , address=185.167.204.206:4067 , piece_id=XWH6ZTD6S3TN6TIFN5S4ZIBNR44VOYCB7UCKXOMBBM2MPMNSXUGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:24.985281962 (103h22m22.582052989s) [noise=true , address=185.167.204.205:4007 , piece_id=GDYOPUY5PJFCYLU7OYPBVTW424GT7FP77TMRYKYNKUWXZS6ENAYQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:30.233899651 (103h22m27.830670686s) [noise=true , address=185.167.204.205:4020 , piece_id=5GIMRHZGF2ZEFWLBL4U6KF4DKZ53JZ7PNWB4XTEQW5XCESHNZY5A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:23:17.302478912 (103h23m14.899249945s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection
Mar 10 18:23:17.302510476 (103h23m14.899281492s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 159.223.143.39:39206->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:23:17.302719428 (103h23m14.89949046s) [noise=true , address=185.167.204.206:4058 , piece_id=LZ6PUPONPGDVBLY3IIUUFUZENZTSTWNMCBEK4GVI3Z2RQXICZSDQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: expected piece hash; context deadline exceeded; EOF
Mar 10 18:23:17.302761551 (103h23m14.899532565s) [seg_pos={0 0}, upload=115702]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:23:17.401134721 (103h23m14.997905738s) [upload=115702]: Done uploading segments...
Mar 10 18:23:17.401207170 (103h23m14.997978196s) [upload=115702]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.273506915s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...................................................X........] duration:
701.166204ms trace:
https://jaeger.dev.storj.io/trace/33e9eb6cbded500c
FAIL (2026-03-10 18:34:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 701.266752ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
600.330365ms trace:
https://jaeger.dev.storj.io/trace/62ea03c3a72aaf43
FAIL (2026-03-10 18:34:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 600.379692ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....................................................X.......] duration:
560.674735ms trace:
https://jaeger.dev.storj.io/trace/33e9eb6cbded500c
FAIL (2026-03-10 18:34:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 560.701166ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [............................................X..X............] duration:
2.547844162s trace:
https://jaeger.dev.storj.io/trace/62b857fbddd8c134
FAIL (2026-03-10 18:33:44 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
TJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.678723726 (103h22m15.275494752s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679171303 (103h22m15.275942329s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679573269 (103h22m15.276344303s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680028932 (103h22m15.27679996s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680473046 (103h22m15.277244178s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680982200 (103h22m15.277753227s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681478081 (103h22m15.27824911s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681895554 (103h22m15.278666585s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682340386 (103h22m15.279111418s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682758716 (103h22m15.279529749s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.683342542 (103h22m15.280113571s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:18.102504696 (103h22m15.699275711s) [upload=115702]: Next returned nil segment
Mar 10 18:22:18.102520582 (103h22m15.699291606s) [upload=115702]: Waiting for error group managing segments...
Mar 10 18:22:18.286488354 (103h22m15.883259367s) [noise=true , address=185.167.204.206:4037 , piece_id=TEYUSWWSWXEKS2TUKAXH7NXD57TQKEZLPS5IOYQVC54ZTY2LEHGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.291859985 (103h22m15.888631012s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.297964517 (103h22m15.894735561s) [noise=true , address=89.149.201.147:4005 , piece_id=CFTY5WE3SZLKOPH6KT4ZD3M2A7ZCIOH7BCUUPGTYM6NRFNJC2BZA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.307699767 (103h22m15.904470781s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.312994778 (103h22m15.909765791s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.319374438 (103h22m15.916145452s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.341507663 (103h22m15.938278696s) [noise=true , address=185.167.204.206:4054 , piece_id=GCJ4ZXR6YMUGI5BG2XOYP4FKTTTSPGVZ5UYS4BOMALGHCSBFQO7Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.351324302 (103h22m15.948095328s) [noise=true , address=81.171.14.70:4032 , piece_id=GPOUM37IH7EL7YANALUZPAQOQ2MHAFOZSVWM7VLDPCNRODOKOO5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.354375553 (103h22m15.951146587s) [noise=true , address=185.167.204.205:4032 , piece_id=YA3TR364FWAMYKQUROV4PYQWTGIXM7SLAUM3BN2FRVBW3MVILHDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.355210000 (103h22m15.951981024s) [noise=true , address=89.149.201.147:4020 , piece_id=AGOZMT6EYQKSFRYOJW4QAV2NMSPM4Q5LAI2SA276EKYK4X5ZGP5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.364878440 (103h22m15.961649464s) [noise=true , address=185.167.204.206:4039 , piece_id=4LSO5WGH5LQ5NIDHOHOACKVAYY37JEZHAUX66MEJ4FIGZ5LWDRFA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.366270355 (103h22m15.963041377s) [noise=true , address=185.167.204.206:4049 , piece_id=JPGUJCUADQNZU5KJAHY4FXUTKJVSEEGMZQ7HNUEWOIWYBJ3LYPEQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.383218910 (103h22m15.979989919s) [noise=true , address=185.167.204.205:4025 , piece_id=2BA453RCULDYAHVTMMVFTPT5MRC4VJNBKK3IHUOLSOMERCHA6LKA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.385734313 (103h22m15.982505326s) [noise=true , address=185.167.204.206:4062 , piece_id=LOWNTJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.386456163 (103h22m15.983227177s) [noise=true , address=185.167.204.206:4071 , piece_id=XT7Y2BSAL4MSA4P6GXDWGQ3R7TGBZT4ON7KAJNVUMBHYPPYU4YTA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389060733 (103h22m15.985831747s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389075564 (103h22m15.985846595s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389787069 (103h22m15.986558092s) [noise=true , address=185.167.204.205:4013 , piece_id=ZYGKKFJQRQZZVEO7DQ67E3ILC66TO5EQBXBAFMKRS2CMKKOKLLVA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.392844303 (103h22m15.989615315s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.395040677 (103h22m15.99181169s) [noise=true , address=185.167.204.206:4053 , piece_id=D72PWJW544HPN6C4C6YIIQBUP7ZIWIN4VCTSYRLGUSCIEUHBWMQA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397224921 (103h22m15.993995937s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397305588 (103h22m15.994076606s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.423243063 (103h22m16.020014089s) [noise=true , address=185.167.204.206:4067 , piece_id=XWH6ZTD6S3TN6TIFN5S4ZIBNR44VOYCB7UCKXOMBBM2MPMNSXUGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:24.985281962 (103h22m22.582052989s) [noise=true , address=185.167.204.205:4007 , piece_id=GDYOPUY5PJFCYLU7OYPBVTW424GT7FP77TMRYKYNKUWXZS6ENAYQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:30.233899651 (103h22m27.830670686s) [noise=true , address=185.167.204.205:4020 , piece_id=5GIMRHZGF2ZEFWLBL4U6KF4DKZ53JZ7PNWB4XTEQW5XCESHNZY5A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:23:17.302478912 (103h23m14.899249945s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection
Mar 10 18:23:17.302510476 (103h23m14.899281492s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 159.223.143.39:39206->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:23:17.302719428 (103h23m14.89949046s) [noise=true , address=185.167.204.206:4058 , piece_id=LZ6PUPONPGDVBLY3IIUUFUZENZTSTWNMCBEK4GVI3Z2RQXICZSDQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: expected piece hash; context deadline exceeded; EOF
Mar 10 18:23:17.302761551 (103h23m14.899532565s) [seg_pos={0 0}, upload=115702]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:23:17.401134721 (103h23m14.997905738s) [upload=115702]: Done uploading segments...
Mar 10 18:23:17.401207170 (103h23m14.997978196s) [upload=115702]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.547871047s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
640.552081ms trace:
https://jaeger.dev.storj.io/trace/62ea03c3a72aaf43
FAIL (2026-03-10 18:33:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 640.632768ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................................................X......] duration:
510.60675ms trace:
https://jaeger.dev.storj.io/trace/33e9eb6cbded500c
FAIL (2026-03-10 18:33:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 510.710907ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
839.96507ms trace:
https://jaeger.dev.storj.io/trace/62ea03c3a72aaf43
FAIL (2026-03-10 18:32:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 839.995542ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.............................................X..X...........] duration:
2.55091915s trace:
https://jaeger.dev.storj.io/trace/62b857fbddd8c134
FAIL (2026-03-10 18:32:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
TJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.678723726 (103h22m15.275494752s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679171303 (103h22m15.275942329s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679573269 (103h22m15.276344303s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680028932 (103h22m15.27679996s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680473046 (103h22m15.277244178s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680982200 (103h22m15.277753227s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681478081 (103h22m15.27824911s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681895554 (103h22m15.278666585s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682340386 (103h22m15.279111418s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682758716 (103h22m15.279529749s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.683342542 (103h22m15.280113571s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:18.102504696 (103h22m15.699275711s) [upload=115702]: Next returned nil segment
Mar 10 18:22:18.102520582 (103h22m15.699291606s) [upload=115702]: Waiting for error group managing segments...
Mar 10 18:22:18.286488354 (103h22m15.883259367s) [noise=true , address=185.167.204.206:4037 , piece_id=TEYUSWWSWXEKS2TUKAXH7NXD57TQKEZLPS5IOYQVC54ZTY2LEHGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.291859985 (103h22m15.888631012s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.297964517 (103h22m15.894735561s) [noise=true , address=89.149.201.147:4005 , piece_id=CFTY5WE3SZLKOPH6KT4ZD3M2A7ZCIOH7BCUUPGTYM6NRFNJC2BZA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.307699767 (103h22m15.904470781s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.312994778 (103h22m15.909765791s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.319374438 (103h22m15.916145452s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.341507663 (103h22m15.938278696s) [noise=true , address=185.167.204.206:4054 , piece_id=GCJ4ZXR6YMUGI5BG2XOYP4FKTTTSPGVZ5UYS4BOMALGHCSBFQO7Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.351324302 (103h22m15.948095328s) [noise=true , address=81.171.14.70:4032 , piece_id=GPOUM37IH7EL7YANALUZPAQOQ2MHAFOZSVWM7VLDPCNRODOKOO5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.354375553 (103h22m15.951146587s) [noise=true , address=185.167.204.205:4032 , piece_id=YA3TR364FWAMYKQUROV4PYQWTGIXM7SLAUM3BN2FRVBW3MVILHDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.355210000 (103h22m15.951981024s) [noise=true , address=89.149.201.147:4020 , piece_id=AGOZMT6EYQKSFRYOJW4QAV2NMSPM4Q5LAI2SA276EKYK4X5ZGP5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.364878440 (103h22m15.961649464s) [noise=true , address=185.167.204.206:4039 , piece_id=4LSO5WGH5LQ5NIDHOHOACKVAYY37JEZHAUX66MEJ4FIGZ5LWDRFA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.366270355 (103h22m15.963041377s) [noise=true , address=185.167.204.206:4049 , piece_id=JPGUJCUADQNZU5KJAHY4FXUTKJVSEEGMZQ7HNUEWOIWYBJ3LYPEQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.383218910 (103h22m15.979989919s) [noise=true , address=185.167.204.205:4025 , piece_id=2BA453RCULDYAHVTMMVFTPT5MRC4VJNBKK3IHUOLSOMERCHA6LKA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.385734313 (103h22m15.982505326s) [noise=true , address=185.167.204.206:4062 , piece_id=LOWNTJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.386456163 (103h22m15.983227177s) [noise=true , address=185.167.204.206:4071 , piece_id=XT7Y2BSAL4MSA4P6GXDWGQ3R7TGBZT4ON7KAJNVUMBHYPPYU4YTA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389060733 (103h22m15.985831747s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389075564 (103h22m15.985846595s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389787069 (103h22m15.986558092s) [noise=true , address=185.167.204.205:4013 , piece_id=ZYGKKFJQRQZZVEO7DQ67E3ILC66TO5EQBXBAFMKRS2CMKKOKLLVA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.392844303 (103h22m15.989615315s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.395040677 (103h22m15.99181169s) [noise=true , address=185.167.204.206:4053 , piece_id=D72PWJW544HPN6C4C6YIIQBUP7ZIWIN4VCTSYRLGUSCIEUHBWMQA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397224921 (103h22m15.993995937s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397305588 (103h22m15.994076606s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.423243063 (103h22m16.020014089s) [noise=true , address=185.167.204.206:4067 , piece_id=XWH6ZTD6S3TN6TIFN5S4ZIBNR44VOYCB7UCKXOMBBM2MPMNSXUGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:24.985281962 (103h22m22.582052989s) [noise=true , address=185.167.204.205:4007 , piece_id=GDYOPUY5PJFCYLU7OYPBVTW424GT7FP77TMRYKYNKUWXZS6ENAYQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:30.233899651 (103h22m27.830670686s) [noise=true , address=185.167.204.205:4020 , piece_id=5GIMRHZGF2ZEFWLBL4U6KF4DKZ53JZ7PNWB4XTEQW5XCESHNZY5A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:23:17.302478912 (103h23m14.899249945s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection
Mar 10 18:23:17.302510476 (103h23m14.899281492s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 159.223.143.39:39206->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:23:17.302719428 (103h23m14.89949046s) [noise=true , address=185.167.204.206:4058 , piece_id=LZ6PUPONPGDVBLY3IIUUFUZENZTSTWNMCBEK4GVI3Z2RQXICZSDQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: expected piece hash; context deadline exceeded; EOF
Mar 10 18:23:17.302761551 (103h23m14.899532565s) [seg_pos={0 0}, upload=115702]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:23:17.401134721 (103h23m14.997905738s) [upload=115702]: Done uploading segments...
Mar 10 18:23:17.401207170 (103h23m14.997978196s) [upload=115702]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.550979552s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................................................X.....] duration:
539.375157ms trace:
https://jaeger.dev.storj.io/trace/33e9eb6cbded500c
FAIL (2026-03-10 18:32:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 539.415862ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
620.233341ms trace:
https://jaeger.dev.storj.io/trace/62ea03c3a72aaf43
FAIL (2026-03-10 18:32:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 620.321091ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......................................................X....] duration:
527.395436ms trace:
https://jaeger.dev.storj.io/trace/33e9eb6cbded500c
FAIL (2026-03-10 18:31:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 527.54545ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
588.603231ms trace:
https://jaeger.dev.storj.io/trace/62ea03c3a72aaf43
FAIL (2026-03-10 18:31:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 588.708259ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..............................................X..X..........] duration:
2.563731832s trace:
https://jaeger.dev.storj.io/trace/62b857fbddd8c134
FAIL (2026-03-10 18:31:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
TJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.678723726 (103h22m15.275494752s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679171303 (103h22m15.275942329s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679573269 (103h22m15.276344303s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680028932 (103h22m15.27679996s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680473046 (103h22m15.277244178s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680982200 (103h22m15.277753227s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681478081 (103h22m15.27824911s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681895554 (103h22m15.278666585s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682340386 (103h22m15.279111418s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682758716 (103h22m15.279529749s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.683342542 (103h22m15.280113571s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:18.102504696 (103h22m15.699275711s) [upload=115702]: Next returned nil segment
Mar 10 18:22:18.102520582 (103h22m15.699291606s) [upload=115702]: Waiting for error group managing segments...
Mar 10 18:22:18.286488354 (103h22m15.883259367s) [noise=true , address=185.167.204.206:4037 , piece_id=TEYUSWWSWXEKS2TUKAXH7NXD57TQKEZLPS5IOYQVC54ZTY2LEHGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.291859985 (103h22m15.888631012s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.297964517 (103h22m15.894735561s) [noise=true , address=89.149.201.147:4005 , piece_id=CFTY5WE3SZLKOPH6KT4ZD3M2A7ZCIOH7BCUUPGTYM6NRFNJC2BZA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.307699767 (103h22m15.904470781s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.312994778 (103h22m15.909765791s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.319374438 (103h22m15.916145452s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.341507663 (103h22m15.938278696s) [noise=true , address=185.167.204.206:4054 , piece_id=GCJ4ZXR6YMUGI5BG2XOYP4FKTTTSPGVZ5UYS4BOMALGHCSBFQO7Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.351324302 (103h22m15.948095328s) [noise=true , address=81.171.14.70:4032 , piece_id=GPOUM37IH7EL7YANALUZPAQOQ2MHAFOZSVWM7VLDPCNRODOKOO5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.354375553 (103h22m15.951146587s) [noise=true , address=185.167.204.205:4032 , piece_id=YA3TR364FWAMYKQUROV4PYQWTGIXM7SLAUM3BN2FRVBW3MVILHDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.355210000 (103h22m15.951981024s) [noise=true , address=89.149.201.147:4020 , piece_id=AGOZMT6EYQKSFRYOJW4QAV2NMSPM4Q5LAI2SA276EKYK4X5ZGP5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.364878440 (103h22m15.961649464s) [noise=true , address=185.167.204.206:4039 , piece_id=4LSO5WGH5LQ5NIDHOHOACKVAYY37JEZHAUX66MEJ4FIGZ5LWDRFA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.366270355 (103h22m15.963041377s) [noise=true , address=185.167.204.206:4049 , piece_id=JPGUJCUADQNZU5KJAHY4FXUTKJVSEEGMZQ7HNUEWOIWYBJ3LYPEQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.383218910 (103h22m15.979989919s) [noise=true , address=185.167.204.205:4025 , piece_id=2BA453RCULDYAHVTMMVFTPT5MRC4VJNBKK3IHUOLSOMERCHA6LKA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.385734313 (103h22m15.982505326s) [noise=true , address=185.167.204.206:4062 , piece_id=LOWNTJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.386456163 (103h22m15.983227177s) [noise=true , address=185.167.204.206:4071 , piece_id=XT7Y2BSAL4MSA4P6GXDWGQ3R7TGBZT4ON7KAJNVUMBHYPPYU4YTA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389060733 (103h22m15.985831747s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389075564 (103h22m15.985846595s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389787069 (103h22m15.986558092s) [noise=true , address=185.167.204.205:4013 , piece_id=ZYGKKFJQRQZZVEO7DQ67E3ILC66TO5EQBXBAFMKRS2CMKKOKLLVA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.392844303 (103h22m15.989615315s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.395040677 (103h22m15.99181169s) [noise=true , address=185.167.204.206:4053 , piece_id=D72PWJW544HPN6C4C6YIIQBUP7ZIWIN4VCTSYRLGUSCIEUHBWMQA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397224921 (103h22m15.993995937s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397305588 (103h22m15.994076606s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.423243063 (103h22m16.020014089s) [noise=true , address=185.167.204.206:4067 , piece_id=XWH6ZTD6S3TN6TIFN5S4ZIBNR44VOYCB7UCKXOMBBM2MPMNSXUGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:24.985281962 (103h22m22.582052989s) [noise=true , address=185.167.204.205:4007 , piece_id=GDYOPUY5PJFCYLU7OYPBVTW424GT7FP77TMRYKYNKUWXZS6ENAYQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:30.233899651 (103h22m27.830670686s) [noise=true , address=185.167.204.205:4020 , piece_id=5GIMRHZGF2ZEFWLBL4U6KF4DKZ53JZ7PNWB4XTEQW5XCESHNZY5A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:23:17.302478912 (103h23m14.899249945s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection
Mar 10 18:23:17.302510476 (103h23m14.899281492s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 159.223.143.39:39206->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:23:17.302719428 (103h23m14.89949046s) [noise=true , address=185.167.204.206:4058 , piece_id=LZ6PUPONPGDVBLY3IIUUFUZENZTSTWNMCBEK4GVI3Z2RQXICZSDQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: expected piece hash; context deadline exceeded; EOF
Mar 10 18:23:17.302761551 (103h23m14.899532565s) [seg_pos={0 0}, upload=115702]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:23:17.401134721 (103h23m14.997905738s) [upload=115702]: Done uploading segments...
Mar 10 18:23:17.401207170 (103h23m14.997978196s) [upload=115702]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.563760658s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...............................................X..X.........] duration:
2.175446257s trace:
https://jaeger.dev.storj.io/trace/62b857fbddd8c134
FAIL (2026-03-10 18:30:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
TJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.678723726 (103h22m15.275494752s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679171303 (103h22m15.275942329s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679573269 (103h22m15.276344303s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680028932 (103h22m15.27679996s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680473046 (103h22m15.277244178s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680982200 (103h22m15.277753227s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681478081 (103h22m15.27824911s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681895554 (103h22m15.278666585s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682340386 (103h22m15.279111418s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682758716 (103h22m15.279529749s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.683342542 (103h22m15.280113571s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:18.102504696 (103h22m15.699275711s) [upload=115702]: Next returned nil segment
Mar 10 18:22:18.102520582 (103h22m15.699291606s) [upload=115702]: Waiting for error group managing segments...
Mar 10 18:22:18.286488354 (103h22m15.883259367s) [noise=true , address=185.167.204.206:4037 , piece_id=TEYUSWWSWXEKS2TUKAXH7NXD57TQKEZLPS5IOYQVC54ZTY2LEHGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.291859985 (103h22m15.888631012s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.297964517 (103h22m15.894735561s) [noise=true , address=89.149.201.147:4005 , piece_id=CFTY5WE3SZLKOPH6KT4ZD3M2A7ZCIOH7BCUUPGTYM6NRFNJC2BZA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.307699767 (103h22m15.904470781s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.312994778 (103h22m15.909765791s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.319374438 (103h22m15.916145452s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.341507663 (103h22m15.938278696s) [noise=true , address=185.167.204.206:4054 , piece_id=GCJ4ZXR6YMUGI5BG2XOYP4FKTTTSPGVZ5UYS4BOMALGHCSBFQO7Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.351324302 (103h22m15.948095328s) [noise=true , address=81.171.14.70:4032 , piece_id=GPOUM37IH7EL7YANALUZPAQOQ2MHAFOZSVWM7VLDPCNRODOKOO5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.354375553 (103h22m15.951146587s) [noise=true , address=185.167.204.205:4032 , piece_id=YA3TR364FWAMYKQUROV4PYQWTGIXM7SLAUM3BN2FRVBW3MVILHDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.355210000 (103h22m15.951981024s) [noise=true , address=89.149.201.147:4020 , piece_id=AGOZMT6EYQKSFRYOJW4QAV2NMSPM4Q5LAI2SA276EKYK4X5ZGP5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.364878440 (103h22m15.961649464s) [noise=true , address=185.167.204.206:4039 , piece_id=4LSO5WGH5LQ5NIDHOHOACKVAYY37JEZHAUX66MEJ4FIGZ5LWDRFA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.366270355 (103h22m15.963041377s) [noise=true , address=185.167.204.206:4049 , piece_id=JPGUJCUADQNZU5KJAHY4FXUTKJVSEEGMZQ7HNUEWOIWYBJ3LYPEQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.383218910 (103h22m15.979989919s) [noise=true , address=185.167.204.205:4025 , piece_id=2BA453RCULDYAHVTMMVFTPT5MRC4VJNBKK3IHUOLSOMERCHA6LKA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.385734313 (103h22m15.982505326s) [noise=true , address=185.167.204.206:4062 , piece_id=LOWNTJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.386456163 (103h22m15.983227177s) [noise=true , address=185.167.204.206:4071 , piece_id=XT7Y2BSAL4MSA4P6GXDWGQ3R7TGBZT4ON7KAJNVUMBHYPPYU4YTA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389060733 (103h22m15.985831747s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389075564 (103h22m15.985846595s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389787069 (103h22m15.986558092s) [noise=true , address=185.167.204.205:4013 , piece_id=ZYGKKFJQRQZZVEO7DQ67E3ILC66TO5EQBXBAFMKRS2CMKKOKLLVA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.392844303 (103h22m15.989615315s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.395040677 (103h22m15.99181169s) [noise=true , address=185.167.204.206:4053 , piece_id=D72PWJW544HPN6C4C6YIIQBUP7ZIWIN4VCTSYRLGUSCIEUHBWMQA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397224921 (103h22m15.993995937s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397305588 (103h22m15.994076606s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.423243063 (103h22m16.020014089s) [noise=true , address=185.167.204.206:4067 , piece_id=XWH6ZTD6S3TN6TIFN5S4ZIBNR44VOYCB7UCKXOMBBM2MPMNSXUGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:24.985281962 (103h22m22.582052989s) [noise=true , address=185.167.204.205:4007 , piece_id=GDYOPUY5PJFCYLU7OYPBVTW424GT7FP77TMRYKYNKUWXZS6ENAYQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:30.233899651 (103h22m27.830670686s) [noise=true , address=185.167.204.205:4020 , piece_id=5GIMRHZGF2ZEFWLBL4U6KF4DKZ53JZ7PNWB4XTEQW5XCESHNZY5A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:23:17.302478912 (103h23m14.899249945s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection
Mar 10 18:23:17.302510476 (103h23m14.899281492s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 159.223.143.39:39206->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:23:17.302719428 (103h23m14.89949046s) [noise=true , address=185.167.204.206:4058 , piece_id=LZ6PUPONPGDVBLY3IIUUFUZENZTSTWNMCBEK4GVI3Z2RQXICZSDQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: expected piece hash; context deadline exceeded; EOF
Mar 10 18:23:17.302761551 (103h23m14.899532565s) [seg_pos={0 0}, upload=115702]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:23:17.401134721 (103h23m14.997905738s) [upload=115702]: Done uploading segments...
Mar 10 18:23:17.401207170 (103h23m14.997978196s) [upload=115702]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.175487978s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................................................X...] duration:
524.370003ms trace:
https://jaeger.dev.storj.io/trace/33e9eb6cbded500c
FAIL (2026-03-10 18:30:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 524.821174ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
600.03091ms trace:
https://jaeger.dev.storj.io/trace/62ea03c3a72aaf43
FAIL (2026-03-10 18:30:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 600.064852ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [................................................X..X........] duration:
2.163446451s trace:
https://jaeger.dev.storj.io/trace/62b857fbddd8c134
FAIL (2026-03-10 18:30:26 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
TJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.678723726 (103h22m15.275494752s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679171303 (103h22m15.275942329s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679573269 (103h22m15.276344303s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680028932 (103h22m15.27679996s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680473046 (103h22m15.277244178s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680982200 (103h22m15.277753227s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681478081 (103h22m15.27824911s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681895554 (103h22m15.278666585s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682340386 (103h22m15.279111418s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682758716 (103h22m15.279529749s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.683342542 (103h22m15.280113571s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:18.102504696 (103h22m15.699275711s) [upload=115702]: Next returned nil segment
Mar 10 18:22:18.102520582 (103h22m15.699291606s) [upload=115702]: Waiting for error group managing segments...
Mar 10 18:22:18.286488354 (103h22m15.883259367s) [noise=true , address=185.167.204.206:4037 , piece_id=TEYUSWWSWXEKS2TUKAXH7NXD57TQKEZLPS5IOYQVC54ZTY2LEHGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.291859985 (103h22m15.888631012s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.297964517 (103h22m15.894735561s) [noise=true , address=89.149.201.147:4005 , piece_id=CFTY5WE3SZLKOPH6KT4ZD3M2A7ZCIOH7BCUUPGTYM6NRFNJC2BZA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.307699767 (103h22m15.904470781s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.312994778 (103h22m15.909765791s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.319374438 (103h22m15.916145452s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.341507663 (103h22m15.938278696s) [noise=true , address=185.167.204.206:4054 , piece_id=GCJ4ZXR6YMUGI5BG2XOYP4FKTTTSPGVZ5UYS4BOMALGHCSBFQO7Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.351324302 (103h22m15.948095328s) [noise=true , address=81.171.14.70:4032 , piece_id=GPOUM37IH7EL7YANALUZPAQOQ2MHAFOZSVWM7VLDPCNRODOKOO5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.354375553 (103h22m15.951146587s) [noise=true , address=185.167.204.205:4032 , piece_id=YA3TR364FWAMYKQUROV4PYQWTGIXM7SLAUM3BN2FRVBW3MVILHDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.355210000 (103h22m15.951981024s) [noise=true , address=89.149.201.147:4020 , piece_id=AGOZMT6EYQKSFRYOJW4QAV2NMSPM4Q5LAI2SA276EKYK4X5ZGP5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.364878440 (103h22m15.961649464s) [noise=true , address=185.167.204.206:4039 , piece_id=4LSO5WGH5LQ5NIDHOHOACKVAYY37JEZHAUX66MEJ4FIGZ5LWDRFA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.366270355 (103h22m15.963041377s) [noise=true , address=185.167.204.206:4049 , piece_id=JPGUJCUADQNZU5KJAHY4FXUTKJVSEEGMZQ7HNUEWOIWYBJ3LYPEQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.383218910 (103h22m15.979989919s) [noise=true , address=185.167.204.205:4025 , piece_id=2BA453RCULDYAHVTMMVFTPT5MRC4VJNBKK3IHUOLSOMERCHA6LKA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.385734313 (103h22m15.982505326s) [noise=true , address=185.167.204.206:4062 , piece_id=LOWNTJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.386456163 (103h22m15.983227177s) [noise=true , address=185.167.204.206:4071 , piece_id=XT7Y2BSAL4MSA4P6GXDWGQ3R7TGBZT4ON7KAJNVUMBHYPPYU4YTA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389060733 (103h22m15.985831747s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389075564 (103h22m15.985846595s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389787069 (103h22m15.986558092s) [noise=true , address=185.167.204.205:4013 , piece_id=ZYGKKFJQRQZZVEO7DQ67E3ILC66TO5EQBXBAFMKRS2CMKKOKLLVA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.392844303 (103h22m15.989615315s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.395040677 (103h22m15.99181169s) [noise=true , address=185.167.204.206:4053 , piece_id=D72PWJW544HPN6C4C6YIIQBUP7ZIWIN4VCTSYRLGUSCIEUHBWMQA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397224921 (103h22m15.993995937s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397305588 (103h22m15.994076606s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.423243063 (103h22m16.020014089s) [noise=true , address=185.167.204.206:4067 , piece_id=XWH6ZTD6S3TN6TIFN5S4ZIBNR44VOYCB7UCKXOMBBM2MPMNSXUGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:24.985281962 (103h22m22.582052989s) [noise=true , address=185.167.204.205:4007 , piece_id=GDYOPUY5PJFCYLU7OYPBVTW424GT7FP77TMRYKYNKUWXZS6ENAYQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:30.233899651 (103h22m27.830670686s) [noise=true , address=185.167.204.205:4020 , piece_id=5GIMRHZGF2ZEFWLBL4U6KF4DKZ53JZ7PNWB4XTEQW5XCESHNZY5A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:23:17.302478912 (103h23m14.899249945s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection
Mar 10 18:23:17.302510476 (103h23m14.899281492s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 159.223.143.39:39206->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:23:17.302719428 (103h23m14.89949046s) [noise=true , address=185.167.204.206:4058 , piece_id=LZ6PUPONPGDVBLY3IIUUFUZENZTSTWNMCBEK4GVI3Z2RQXICZSDQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: expected piece hash; context deadline exceeded; EOF
Mar 10 18:23:17.302761551 (103h23m14.899532565s) [seg_pos={0 0}, upload=115702]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:23:17.401134721 (103h23m14.997905738s) [upload=115702]: Done uploading segments...
Mar 10 18:23:17.401207170 (103h23m14.997978196s) [upload=115702]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.163480346s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................................................X..] duration:
578.350777ms trace:
https://jaeger.dev.storj.io/trace/33e9eb6cbded500c
FAIL (2026-03-10 18:29:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 578.374901ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
828.37286ms trace:
https://jaeger.dev.storj.io/trace/62ea03c3a72aaf43
FAIL (2026-03-10 18:29:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 828.406383ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.001635339s trace:
https://jaeger.dev.storj.io/trace/62ea03c3a72aaf43
FAIL (2026-03-10 18:29:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001661961s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................................................X.] duration:
473.934511ms trace:
https://jaeger.dev.storj.io/trace/33e9eb6cbded500c
FAIL (2026-03-10 18:29:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 473.969725ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.................................................X..X.......] duration:
2.360978577s trace:
https://jaeger.dev.storj.io/trace/62b857fbddd8c134
FAIL (2026-03-10 18:29:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
TJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.678723726 (103h22m15.275494752s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679171303 (103h22m15.275942329s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679573269 (103h22m15.276344303s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680028932 (103h22m15.27679996s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680473046 (103h22m15.277244178s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680982200 (103h22m15.277753227s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681478081 (103h22m15.27824911s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681895554 (103h22m15.278666585s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682340386 (103h22m15.279111418s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682758716 (103h22m15.279529749s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.683342542 (103h22m15.280113571s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:18.102504696 (103h22m15.699275711s) [upload=115702]: Next returned nil segment
Mar 10 18:22:18.102520582 (103h22m15.699291606s) [upload=115702]: Waiting for error group managing segments...
Mar 10 18:22:18.286488354 (103h22m15.883259367s) [noise=true , address=185.167.204.206:4037 , piece_id=TEYUSWWSWXEKS2TUKAXH7NXD57TQKEZLPS5IOYQVC54ZTY2LEHGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.291859985 (103h22m15.888631012s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.297964517 (103h22m15.894735561s) [noise=true , address=89.149.201.147:4005 , piece_id=CFTY5WE3SZLKOPH6KT4ZD3M2A7ZCIOH7BCUUPGTYM6NRFNJC2BZA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.307699767 (103h22m15.904470781s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.312994778 (103h22m15.909765791s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.319374438 (103h22m15.916145452s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.341507663 (103h22m15.938278696s) [noise=true , address=185.167.204.206:4054 , piece_id=GCJ4ZXR6YMUGI5BG2XOYP4FKTTTSPGVZ5UYS4BOMALGHCSBFQO7Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.351324302 (103h22m15.948095328s) [noise=true , address=81.171.14.70:4032 , piece_id=GPOUM37IH7EL7YANALUZPAQOQ2MHAFOZSVWM7VLDPCNRODOKOO5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.354375553 (103h22m15.951146587s) [noise=true , address=185.167.204.205:4032 , piece_id=YA3TR364FWAMYKQUROV4PYQWTGIXM7SLAUM3BN2FRVBW3MVILHDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.355210000 (103h22m15.951981024s) [noise=true , address=89.149.201.147:4020 , piece_id=AGOZMT6EYQKSFRYOJW4QAV2NMSPM4Q5LAI2SA276EKYK4X5ZGP5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.364878440 (103h22m15.961649464s) [noise=true , address=185.167.204.206:4039 , piece_id=4LSO5WGH5LQ5NIDHOHOACKVAYY37JEZHAUX66MEJ4FIGZ5LWDRFA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.366270355 (103h22m15.963041377s) [noise=true , address=185.167.204.206:4049 , piece_id=JPGUJCUADQNZU5KJAHY4FXUTKJVSEEGMZQ7HNUEWOIWYBJ3LYPEQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.383218910 (103h22m15.979989919s) [noise=true , address=185.167.204.205:4025 , piece_id=2BA453RCULDYAHVTMMVFTPT5MRC4VJNBKK3IHUOLSOMERCHA6LKA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.385734313 (103h22m15.982505326s) [noise=true , address=185.167.204.206:4062 , piece_id=LOWNTJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.386456163 (103h22m15.983227177s) [noise=true , address=185.167.204.206:4071 , piece_id=XT7Y2BSAL4MSA4P6GXDWGQ3R7TGBZT4ON7KAJNVUMBHYPPYU4YTA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389060733 (103h22m15.985831747s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389075564 (103h22m15.985846595s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389787069 (103h22m15.986558092s) [noise=true , address=185.167.204.205:4013 , piece_id=ZYGKKFJQRQZZVEO7DQ67E3ILC66TO5EQBXBAFMKRS2CMKKOKLLVA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.392844303 (103h22m15.989615315s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.395040677 (103h22m15.99181169s) [noise=true , address=185.167.204.206:4053 , piece_id=D72PWJW544HPN6C4C6YIIQBUP7ZIWIN4VCTSYRLGUSCIEUHBWMQA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397224921 (103h22m15.993995937s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397305588 (103h22m15.994076606s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.423243063 (103h22m16.020014089s) [noise=true , address=185.167.204.206:4067 , piece_id=XWH6ZTD6S3TN6TIFN5S4ZIBNR44VOYCB7UCKXOMBBM2MPMNSXUGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:24.985281962 (103h22m22.582052989s) [noise=true , address=185.167.204.205:4007 , piece_id=GDYOPUY5PJFCYLU7OYPBVTW424GT7FP77TMRYKYNKUWXZS6ENAYQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:30.233899651 (103h22m27.830670686s) [noise=true , address=185.167.204.205:4020 , piece_id=5GIMRHZGF2ZEFWLBL4U6KF4DKZ53JZ7PNWB4XTEQW5XCESHNZY5A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:23:17.302478912 (103h23m14.899249945s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection
Mar 10 18:23:17.302510476 (103h23m14.899281492s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 159.223.143.39:39206->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:23:17.302719428 (103h23m14.89949046s) [noise=true , address=185.167.204.206:4058 , piece_id=LZ6PUPONPGDVBLY3IIUUFUZENZTSTWNMCBEK4GVI3Z2RQXICZSDQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: expected piece hash; context deadline exceeded; EOF
Mar 10 18:23:17.302761551 (103h23m14.899532565s) [seg_pos={0 0}, upload=115702]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:23:17.401134721 (103h23m14.997905738s) [upload=115702]: Done uploading segments...
Mar 10 18:23:17.401207170 (103h23m14.997978196s) [upload=115702]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.36100597s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................................................X] duration:
5.0014253s trace:
https://jaeger.dev.storj.io/trace/33e9eb6cbded500c
FAIL (2026-03-10 18:29:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001450183s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..................................................X..X......] duration:
2.65170451s trace:
https://jaeger.dev.storj.io/trace/62b857fbddd8c134
FAIL (2026-03-10 18:27:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
TJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.678723726 (103h22m15.275494752s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679171303 (103h22m15.275942329s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679573269 (103h22m15.276344303s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680028932 (103h22m15.27679996s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680473046 (103h22m15.277244178s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680982200 (103h22m15.277753227s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681478081 (103h22m15.27824911s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681895554 (103h22m15.278666585s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682340386 (103h22m15.279111418s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682758716 (103h22m15.279529749s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.683342542 (103h22m15.280113571s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:18.102504696 (103h22m15.699275711s) [upload=115702]: Next returned nil segment
Mar 10 18:22:18.102520582 (103h22m15.699291606s) [upload=115702]: Waiting for error group managing segments...
Mar 10 18:22:18.286488354 (103h22m15.883259367s) [noise=true , address=185.167.204.206:4037 , piece_id=TEYUSWWSWXEKS2TUKAXH7NXD57TQKEZLPS5IOYQVC54ZTY2LEHGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.291859985 (103h22m15.888631012s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.297964517 (103h22m15.894735561s) [noise=true , address=89.149.201.147:4005 , piece_id=CFTY5WE3SZLKOPH6KT4ZD3M2A7ZCIOH7BCUUPGTYM6NRFNJC2BZA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.307699767 (103h22m15.904470781s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.312994778 (103h22m15.909765791s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.319374438 (103h22m15.916145452s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.341507663 (103h22m15.938278696s) [noise=true , address=185.167.204.206:4054 , piece_id=GCJ4ZXR6YMUGI5BG2XOYP4FKTTTSPGVZ5UYS4BOMALGHCSBFQO7Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.351324302 (103h22m15.948095328s) [noise=true , address=81.171.14.70:4032 , piece_id=GPOUM37IH7EL7YANALUZPAQOQ2MHAFOZSVWM7VLDPCNRODOKOO5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.354375553 (103h22m15.951146587s) [noise=true , address=185.167.204.205:4032 , piece_id=YA3TR364FWAMYKQUROV4PYQWTGIXM7SLAUM3BN2FRVBW3MVILHDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.355210000 (103h22m15.951981024s) [noise=true , address=89.149.201.147:4020 , piece_id=AGOZMT6EYQKSFRYOJW4QAV2NMSPM4Q5LAI2SA276EKYK4X5ZGP5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.364878440 (103h22m15.961649464s) [noise=true , address=185.167.204.206:4039 , piece_id=4LSO5WGH5LQ5NIDHOHOACKVAYY37JEZHAUX66MEJ4FIGZ5LWDRFA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.366270355 (103h22m15.963041377s) [noise=true , address=185.167.204.206:4049 , piece_id=JPGUJCUADQNZU5KJAHY4FXUTKJVSEEGMZQ7HNUEWOIWYBJ3LYPEQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.383218910 (103h22m15.979989919s) [noise=true , address=185.167.204.205:4025 , piece_id=2BA453RCULDYAHVTMMVFTPT5MRC4VJNBKK3IHUOLSOMERCHA6LKA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.385734313 (103h22m15.982505326s) [noise=true , address=185.167.204.206:4062 , piece_id=LOWNTJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.386456163 (103h22m15.983227177s) [noise=true , address=185.167.204.206:4071 , piece_id=XT7Y2BSAL4MSA4P6GXDWGQ3R7TGBZT4ON7KAJNVUMBHYPPYU4YTA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389060733 (103h22m15.985831747s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389075564 (103h22m15.985846595s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389787069 (103h22m15.986558092s) [noise=true , address=185.167.204.205:4013 , piece_id=ZYGKKFJQRQZZVEO7DQ67E3ILC66TO5EQBXBAFMKRS2CMKKOKLLVA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.392844303 (103h22m15.989615315s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.395040677 (103h22m15.99181169s) [noise=true , address=185.167.204.206:4053 , piece_id=D72PWJW544HPN6C4C6YIIQBUP7ZIWIN4VCTSYRLGUSCIEUHBWMQA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397224921 (103h22m15.993995937s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397305588 (103h22m15.994076606s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.423243063 (103h22m16.020014089s) [noise=true , address=185.167.204.206:4067 , piece_id=XWH6ZTD6S3TN6TIFN5S4ZIBNR44VOYCB7UCKXOMBBM2MPMNSXUGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:24.985281962 (103h22m22.582052989s) [noise=true , address=185.167.204.205:4007 , piece_id=GDYOPUY5PJFCYLU7OYPBVTW424GT7FP77TMRYKYNKUWXZS6ENAYQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:30.233899651 (103h22m27.830670686s) [noise=true , address=185.167.204.205:4020 , piece_id=5GIMRHZGF2ZEFWLBL4U6KF4DKZ53JZ7PNWB4XTEQW5XCESHNZY5A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:23:17.302478912 (103h23m14.899249945s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection
Mar 10 18:23:17.302510476 (103h23m14.899281492s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 159.223.143.39:39206->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:23:17.302719428 (103h23m14.89949046s) [noise=true , address=185.167.204.206:4058 , piece_id=LZ6PUPONPGDVBLY3IIUUFUZENZTSTWNMCBEK4GVI3Z2RQXICZSDQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: expected piece hash; context deadline exceeded; EOF
Mar 10 18:23:17.302761551 (103h23m14.899532565s) [seg_pos={0 0}, upload=115702]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:23:17.401134721 (103h23m14.997905738s) [upload=115702]: Done uploading segments...
Mar 10 18:23:17.401207170 (103h23m14.997978196s) [upload=115702]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.651730288s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...................................................X..X.....] duration:
2.542376732s trace:
https://jaeger.dev.storj.io/trace/62b857fbddd8c134
FAIL (2026-03-10 18:26:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
TJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.678723726 (103h22m15.275494752s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679171303 (103h22m15.275942329s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679573269 (103h22m15.276344303s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680028932 (103h22m15.27679996s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680473046 (103h22m15.277244178s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680982200 (103h22m15.277753227s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681478081 (103h22m15.27824911s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681895554 (103h22m15.278666585s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682340386 (103h22m15.279111418s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682758716 (103h22m15.279529749s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.683342542 (103h22m15.280113571s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:18.102504696 (103h22m15.699275711s) [upload=115702]: Next returned nil segment
Mar 10 18:22:18.102520582 (103h22m15.699291606s) [upload=115702]: Waiting for error group managing segments...
Mar 10 18:22:18.286488354 (103h22m15.883259367s) [noise=true , address=185.167.204.206:4037 , piece_id=TEYUSWWSWXEKS2TUKAXH7NXD57TQKEZLPS5IOYQVC54ZTY2LEHGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.291859985 (103h22m15.888631012s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.297964517 (103h22m15.894735561s) [noise=true , address=89.149.201.147:4005 , piece_id=CFTY5WE3SZLKOPH6KT4ZD3M2A7ZCIOH7BCUUPGTYM6NRFNJC2BZA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.307699767 (103h22m15.904470781s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.312994778 (103h22m15.909765791s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.319374438 (103h22m15.916145452s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.341507663 (103h22m15.938278696s) [noise=true , address=185.167.204.206:4054 , piece_id=GCJ4ZXR6YMUGI5BG2XOYP4FKTTTSPGVZ5UYS4BOMALGHCSBFQO7Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.351324302 (103h22m15.948095328s) [noise=true , address=81.171.14.70:4032 , piece_id=GPOUM37IH7EL7YANALUZPAQOQ2MHAFOZSVWM7VLDPCNRODOKOO5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.354375553 (103h22m15.951146587s) [noise=true , address=185.167.204.205:4032 , piece_id=YA3TR364FWAMYKQUROV4PYQWTGIXM7SLAUM3BN2FRVBW3MVILHDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.355210000 (103h22m15.951981024s) [noise=true , address=89.149.201.147:4020 , piece_id=AGOZMT6EYQKSFRYOJW4QAV2NMSPM4Q5LAI2SA276EKYK4X5ZGP5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.364878440 (103h22m15.961649464s) [noise=true , address=185.167.204.206:4039 , piece_id=4LSO5WGH5LQ5NIDHOHOACKVAYY37JEZHAUX66MEJ4FIGZ5LWDRFA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.366270355 (103h22m15.963041377s) [noise=true , address=185.167.204.206:4049 , piece_id=JPGUJCUADQNZU5KJAHY4FXUTKJVSEEGMZQ7HNUEWOIWYBJ3LYPEQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.383218910 (103h22m15.979989919s) [noise=true , address=185.167.204.205:4025 , piece_id=2BA453RCULDYAHVTMMVFTPT5MRC4VJNBKK3IHUOLSOMERCHA6LKA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.385734313 (103h22m15.982505326s) [noise=true , address=185.167.204.206:4062 , piece_id=LOWNTJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.386456163 (103h22m15.983227177s) [noise=true , address=185.167.204.206:4071 , piece_id=XT7Y2BSAL4MSA4P6GXDWGQ3R7TGBZT4ON7KAJNVUMBHYPPYU4YTA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389060733 (103h22m15.985831747s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389075564 (103h22m15.985846595s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389787069 (103h22m15.986558092s) [noise=true , address=185.167.204.205:4013 , piece_id=ZYGKKFJQRQZZVEO7DQ67E3ILC66TO5EQBXBAFMKRS2CMKKOKLLVA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.392844303 (103h22m15.989615315s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.395040677 (103h22m15.99181169s) [noise=true , address=185.167.204.206:4053 , piece_id=D72PWJW544HPN6C4C6YIIQBUP7ZIWIN4VCTSYRLGUSCIEUHBWMQA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397224921 (103h22m15.993995937s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397305588 (103h22m15.994076606s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.423243063 (103h22m16.020014089s) [noise=true , address=185.167.204.206:4067 , piece_id=XWH6ZTD6S3TN6TIFN5S4ZIBNR44VOYCB7UCKXOMBBM2MPMNSXUGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:24.985281962 (103h22m22.582052989s) [noise=true , address=185.167.204.205:4007 , piece_id=GDYOPUY5PJFCYLU7OYPBVTW424GT7FP77TMRYKYNKUWXZS6ENAYQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:30.233899651 (103h22m27.830670686s) [noise=true , address=185.167.204.205:4020 , piece_id=5GIMRHZGF2ZEFWLBL4U6KF4DKZ53JZ7PNWB4XTEQW5XCESHNZY5A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:23:17.302478912 (103h23m14.899249945s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection
Mar 10 18:23:17.302510476 (103h23m14.899281492s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 159.223.143.39:39206->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:23:17.302719428 (103h23m14.89949046s) [noise=true , address=185.167.204.206:4058 , piece_id=LZ6PUPONPGDVBLY3IIUUFUZENZTSTWNMCBEK4GVI3Z2RQXICZSDQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: expected piece hash; context deadline exceeded; EOF
Mar 10 18:23:17.302761551 (103h23m14.899532565s) [seg_pos={0 0}, upload=115702]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:23:17.401134721 (103h23m14.997905738s) [upload=115702]: Done uploading segments...
Mar 10 18:23:17.401207170 (103h23m14.997978196s) [upload=115702]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.542417902s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [....................................................X..X....] duration:
11.323545529s trace:
https://jaeger.dev.storj.io/trace/62b857fbddd8c134
FAIL (2026-03-10 18:26:09 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
TJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.678723726 (103h22m15.275494752s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679171303 (103h22m15.275942329s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679573269 (103h22m15.276344303s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680028932 (103h22m15.27679996s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680473046 (103h22m15.277244178s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680982200 (103h22m15.277753227s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681478081 (103h22m15.27824911s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681895554 (103h22m15.278666585s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682340386 (103h22m15.279111418s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682758716 (103h22m15.279529749s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.683342542 (103h22m15.280113571s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:18.102504696 (103h22m15.699275711s) [upload=115702]: Next returned nil segment
Mar 10 18:22:18.102520582 (103h22m15.699291606s) [upload=115702]: Waiting for error group managing segments...
Mar 10 18:22:18.286488354 (103h22m15.883259367s) [noise=true , address=185.167.204.206:4037 , piece_id=TEYUSWWSWXEKS2TUKAXH7NXD57TQKEZLPS5IOYQVC54ZTY2LEHGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.291859985 (103h22m15.888631012s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.297964517 (103h22m15.894735561s) [noise=true , address=89.149.201.147:4005 , piece_id=CFTY5WE3SZLKOPH6KT4ZD3M2A7ZCIOH7BCUUPGTYM6NRFNJC2BZA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.307699767 (103h22m15.904470781s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.312994778 (103h22m15.909765791s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.319374438 (103h22m15.916145452s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.341507663 (103h22m15.938278696s) [noise=true , address=185.167.204.206:4054 , piece_id=GCJ4ZXR6YMUGI5BG2XOYP4FKTTTSPGVZ5UYS4BOMALGHCSBFQO7Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.351324302 (103h22m15.948095328s) [noise=true , address=81.171.14.70:4032 , piece_id=GPOUM37IH7EL7YANALUZPAQOQ2MHAFOZSVWM7VLDPCNRODOKOO5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.354375553 (103h22m15.951146587s) [noise=true , address=185.167.204.205:4032 , piece_id=YA3TR364FWAMYKQUROV4PYQWTGIXM7SLAUM3BN2FRVBW3MVILHDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.355210000 (103h22m15.951981024s) [noise=true , address=89.149.201.147:4020 , piece_id=AGOZMT6EYQKSFRYOJW4QAV2NMSPM4Q5LAI2SA276EKYK4X5ZGP5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.364878440 (103h22m15.961649464s) [noise=true , address=185.167.204.206:4039 , piece_id=4LSO5WGH5LQ5NIDHOHOACKVAYY37JEZHAUX66MEJ4FIGZ5LWDRFA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.366270355 (103h22m15.963041377s) [noise=true , address=185.167.204.206:4049 , piece_id=JPGUJCUADQNZU5KJAHY4FXUTKJVSEEGMZQ7HNUEWOIWYBJ3LYPEQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.383218910 (103h22m15.979989919s) [noise=true , address=185.167.204.205:4025 , piece_id=2BA453RCULDYAHVTMMVFTPT5MRC4VJNBKK3IHUOLSOMERCHA6LKA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.385734313 (103h22m15.982505326s) [noise=true , address=185.167.204.206:4062 , piece_id=LOWNTJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.386456163 (103h22m15.983227177s) [noise=true , address=185.167.204.206:4071 , piece_id=XT7Y2BSAL4MSA4P6GXDWGQ3R7TGBZT4ON7KAJNVUMBHYPPYU4YTA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389060733 (103h22m15.985831747s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389075564 (103h22m15.985846595s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389787069 (103h22m15.986558092s) [noise=true , address=185.167.204.205:4013 , piece_id=ZYGKKFJQRQZZVEO7DQ67E3ILC66TO5EQBXBAFMKRS2CMKKOKLLVA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.392844303 (103h22m15.989615315s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.395040677 (103h22m15.99181169s) [noise=true , address=185.167.204.206:4053 , piece_id=D72PWJW544HPN6C4C6YIIQBUP7ZIWIN4VCTSYRLGUSCIEUHBWMQA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397224921 (103h22m15.993995937s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397305588 (103h22m15.994076606s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.423243063 (103h22m16.020014089s) [noise=true , address=185.167.204.206:4067 , piece_id=XWH6ZTD6S3TN6TIFN5S4ZIBNR44VOYCB7UCKXOMBBM2MPMNSXUGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:24.985281962 (103h22m22.582052989s) [noise=true , address=185.167.204.205:4007 , piece_id=GDYOPUY5PJFCYLU7OYPBVTW424GT7FP77TMRYKYNKUWXZS6ENAYQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:30.233899651 (103h22m27.830670686s) [noise=true , address=185.167.204.205:4020 , piece_id=5GIMRHZGF2ZEFWLBL4U6KF4DKZ53JZ7PNWB4XTEQW5XCESHNZY5A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:23:17.302478912 (103h23m14.899249945s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection
Mar 10 18:23:17.302510476 (103h23m14.899281492s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 159.223.143.39:39206->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:23:17.302719428 (103h23m14.89949046s) [noise=true , address=185.167.204.206:4058 , piece_id=LZ6PUPONPGDVBLY3IIUUFUZENZTSTWNMCBEK4GVI3Z2RQXICZSDQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: expected piece hash; context deadline exceeded; EOF
Mar 10 18:23:17.302761551 (103h23m14.899532565s) [seg_pos={0 0}, upload=115702]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:23:17.401134721 (103h23m14.997905738s) [upload=115702]: Done uploading segments...
Mar 10 18:23:17.401207170 (103h23m14.997978196s) [upload=115702]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 11.323612755s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.....................................................X..X...] duration:
2.254070247s trace:
https://jaeger.dev.storj.io/trace/62b857fbddd8c134
FAIL (2026-03-10 18:24:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
TJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.678723726 (103h22m15.275494752s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679171303 (103h22m15.275942329s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679573269 (103h22m15.276344303s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680028932 (103h22m15.27679996s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680473046 (103h22m15.277244178s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680982200 (103h22m15.277753227s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681478081 (103h22m15.27824911s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681895554 (103h22m15.278666585s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682340386 (103h22m15.279111418s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682758716 (103h22m15.279529749s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.683342542 (103h22m15.280113571s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:18.102504696 (103h22m15.699275711s) [upload=115702]: Next returned nil segment
Mar 10 18:22:18.102520582 (103h22m15.699291606s) [upload=115702]: Waiting for error group managing segments...
Mar 10 18:22:18.286488354 (103h22m15.883259367s) [noise=true , address=185.167.204.206:4037 , piece_id=TEYUSWWSWXEKS2TUKAXH7NXD57TQKEZLPS5IOYQVC54ZTY2LEHGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.291859985 (103h22m15.888631012s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.297964517 (103h22m15.894735561s) [noise=true , address=89.149.201.147:4005 , piece_id=CFTY5WE3SZLKOPH6KT4ZD3M2A7ZCIOH7BCUUPGTYM6NRFNJC2BZA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.307699767 (103h22m15.904470781s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.312994778 (103h22m15.909765791s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.319374438 (103h22m15.916145452s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.341507663 (103h22m15.938278696s) [noise=true , address=185.167.204.206:4054 , piece_id=GCJ4ZXR6YMUGI5BG2XOYP4FKTTTSPGVZ5UYS4BOMALGHCSBFQO7Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.351324302 (103h22m15.948095328s) [noise=true , address=81.171.14.70:4032 , piece_id=GPOUM37IH7EL7YANALUZPAQOQ2MHAFOZSVWM7VLDPCNRODOKOO5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.354375553 (103h22m15.951146587s) [noise=true , address=185.167.204.205:4032 , piece_id=YA3TR364FWAMYKQUROV4PYQWTGIXM7SLAUM3BN2FRVBW3MVILHDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.355210000 (103h22m15.951981024s) [noise=true , address=89.149.201.147:4020 , piece_id=AGOZMT6EYQKSFRYOJW4QAV2NMSPM4Q5LAI2SA276EKYK4X5ZGP5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.364878440 (103h22m15.961649464s) [noise=true , address=185.167.204.206:4039 , piece_id=4LSO5WGH5LQ5NIDHOHOACKVAYY37JEZHAUX66MEJ4FIGZ5LWDRFA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.366270355 (103h22m15.963041377s) [noise=true , address=185.167.204.206:4049 , piece_id=JPGUJCUADQNZU5KJAHY4FXUTKJVSEEGMZQ7HNUEWOIWYBJ3LYPEQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.383218910 (103h22m15.979989919s) [noise=true , address=185.167.204.205:4025 , piece_id=2BA453RCULDYAHVTMMVFTPT5MRC4VJNBKK3IHUOLSOMERCHA6LKA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.385734313 (103h22m15.982505326s) [noise=true , address=185.167.204.206:4062 , piece_id=LOWNTJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.386456163 (103h22m15.983227177s) [noise=true , address=185.167.204.206:4071 , piece_id=XT7Y2BSAL4MSA4P6GXDWGQ3R7TGBZT4ON7KAJNVUMBHYPPYU4YTA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389060733 (103h22m15.985831747s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389075564 (103h22m15.985846595s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389787069 (103h22m15.986558092s) [noise=true , address=185.167.204.205:4013 , piece_id=ZYGKKFJQRQZZVEO7DQ67E3ILC66TO5EQBXBAFMKRS2CMKKOKLLVA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.392844303 (103h22m15.989615315s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.395040677 (103h22m15.99181169s) [noise=true , address=185.167.204.206:4053 , piece_id=D72PWJW544HPN6C4C6YIIQBUP7ZIWIN4VCTSYRLGUSCIEUHBWMQA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397224921 (103h22m15.993995937s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397305588 (103h22m15.994076606s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.423243063 (103h22m16.020014089s) [noise=true , address=185.167.204.206:4067 , piece_id=XWH6ZTD6S3TN6TIFN5S4ZIBNR44VOYCB7UCKXOMBBM2MPMNSXUGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:24.985281962 (103h22m22.582052989s) [noise=true , address=185.167.204.205:4007 , piece_id=GDYOPUY5PJFCYLU7OYPBVTW424GT7FP77TMRYKYNKUWXZS6ENAYQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:30.233899651 (103h22m27.830670686s) [noise=true , address=185.167.204.205:4020 , piece_id=5GIMRHZGF2ZEFWLBL4U6KF4DKZ53JZ7PNWB4XTEQW5XCESHNZY5A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:23:17.302478912 (103h23m14.899249945s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection
Mar 10 18:23:17.302510476 (103h23m14.899281492s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 159.223.143.39:39206->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:23:17.302719428 (103h23m14.89949046s) [noise=true , address=185.167.204.206:4058 , piece_id=LZ6PUPONPGDVBLY3IIUUFUZENZTSTWNMCBEK4GVI3Z2RQXICZSDQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: expected piece hash; context deadline exceeded; EOF
Mar 10 18:23:17.302761551 (103h23m14.899532565s) [seg_pos={0 0}, upload=115702]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:23:17.401134721 (103h23m14.997905738s) [upload=115702]: Done uploading segments...
Mar 10 18:23:17.401207170 (103h23m14.997978196s) [upload=115702]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.254155806s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [......................................................X..X..] duration:
20.743873689s trace:
https://jaeger.dev.storj.io/trace/62b857fbddd8c134
FAIL (2026-03-10 18:24:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
TJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.678723726 (103h22m15.275494752s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679171303 (103h22m15.275942329s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679573269 (103h22m15.276344303s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680028932 (103h22m15.27679996s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680473046 (103h22m15.277244178s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680982200 (103h22m15.277753227s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681478081 (103h22m15.27824911s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681895554 (103h22m15.278666585s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682340386 (103h22m15.279111418s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682758716 (103h22m15.279529749s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.683342542 (103h22m15.280113571s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:18.102504696 (103h22m15.699275711s) [upload=115702]: Next returned nil segment
Mar 10 18:22:18.102520582 (103h22m15.699291606s) [upload=115702]: Waiting for error group managing segments...
Mar 10 18:22:18.286488354 (103h22m15.883259367s) [noise=true , address=185.167.204.206:4037 , piece_id=TEYUSWWSWXEKS2TUKAXH7NXD57TQKEZLPS5IOYQVC54ZTY2LEHGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.291859985 (103h22m15.888631012s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.297964517 (103h22m15.894735561s) [noise=true , address=89.149.201.147:4005 , piece_id=CFTY5WE3SZLKOPH6KT4ZD3M2A7ZCIOH7BCUUPGTYM6NRFNJC2BZA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.307699767 (103h22m15.904470781s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.312994778 (103h22m15.909765791s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.319374438 (103h22m15.916145452s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.341507663 (103h22m15.938278696s) [noise=true , address=185.167.204.206:4054 , piece_id=GCJ4ZXR6YMUGI5BG2XOYP4FKTTTSPGVZ5UYS4BOMALGHCSBFQO7Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.351324302 (103h22m15.948095328s) [noise=true , address=81.171.14.70:4032 , piece_id=GPOUM37IH7EL7YANALUZPAQOQ2MHAFOZSVWM7VLDPCNRODOKOO5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.354375553 (103h22m15.951146587s) [noise=true , address=185.167.204.205:4032 , piece_id=YA3TR364FWAMYKQUROV4PYQWTGIXM7SLAUM3BN2FRVBW3MVILHDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.355210000 (103h22m15.951981024s) [noise=true , address=89.149.201.147:4020 , piece_id=AGOZMT6EYQKSFRYOJW4QAV2NMSPM4Q5LAI2SA276EKYK4X5ZGP5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.364878440 (103h22m15.961649464s) [noise=true , address=185.167.204.206:4039 , piece_id=4LSO5WGH5LQ5NIDHOHOACKVAYY37JEZHAUX66MEJ4FIGZ5LWDRFA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.366270355 (103h22m15.963041377s) [noise=true , address=185.167.204.206:4049 , piece_id=JPGUJCUADQNZU5KJAHY4FXUTKJVSEEGMZQ7HNUEWOIWYBJ3LYPEQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.383218910 (103h22m15.979989919s) [noise=true , address=185.167.204.205:4025 , piece_id=2BA453RCULDYAHVTMMVFTPT5MRC4VJNBKK3IHUOLSOMERCHA6LKA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.385734313 (103h22m15.982505326s) [noise=true , address=185.167.204.206:4062 , piece_id=LOWNTJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.386456163 (103h22m15.983227177s) [noise=true , address=185.167.204.206:4071 , piece_id=XT7Y2BSAL4MSA4P6GXDWGQ3R7TGBZT4ON7KAJNVUMBHYPPYU4YTA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389060733 (103h22m15.985831747s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389075564 (103h22m15.985846595s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389787069 (103h22m15.986558092s) [noise=true , address=185.167.204.205:4013 , piece_id=ZYGKKFJQRQZZVEO7DQ67E3ILC66TO5EQBXBAFMKRS2CMKKOKLLVA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.392844303 (103h22m15.989615315s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.395040677 (103h22m15.99181169s) [noise=true , address=185.167.204.206:4053 , piece_id=D72PWJW544HPN6C4C6YIIQBUP7ZIWIN4VCTSYRLGUSCIEUHBWMQA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397224921 (103h22m15.993995937s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397305588 (103h22m15.994076606s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.423243063 (103h22m16.020014089s) [noise=true , address=185.167.204.206:4067 , piece_id=XWH6ZTD6S3TN6TIFN5S4ZIBNR44VOYCB7UCKXOMBBM2MPMNSXUGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:24.985281962 (103h22m22.582052989s) [noise=true , address=185.167.204.205:4007 , piece_id=GDYOPUY5PJFCYLU7OYPBVTW424GT7FP77TMRYKYNKUWXZS6ENAYQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:30.233899651 (103h22m27.830670686s) [noise=true , address=185.167.204.205:4020 , piece_id=5GIMRHZGF2ZEFWLBL4U6KF4DKZ53JZ7PNWB4XTEQW5XCESHNZY5A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:23:17.302478912 (103h23m14.899249945s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection
Mar 10 18:23:17.302510476 (103h23m14.899281492s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 159.223.143.39:39206->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:23:17.302719428 (103h23m14.89949046s) [noise=true , address=185.167.204.206:4058 , piece_id=LZ6PUPONPGDVBLY3IIUUFUZENZTSTWNMCBEK4GVI3Z2RQXICZSDQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: expected piece hash; context deadline exceeded; EOF
Mar 10 18:23:17.302761551 (103h23m14.899532565s) [seg_pos={0 0}, upload=115702]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:23:17.401134721 (103h23m14.997905738s) [upload=115702]: Done uploading segments...
Mar 10 18:23:17.401207170 (103h23m14.997978196s) [upload=115702]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 20.743898628s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.......................................................X..X.] duration:
28.245622591s trace:
https://jaeger.dev.storj.io/trace/62b857fbddd8c134
FAIL (2026-03-10 18:23:45 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
TJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.678723726 (103h22m15.275494752s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679171303 (103h22m15.275942329s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679573269 (103h22m15.276344303s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680028932 (103h22m15.27679996s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680473046 (103h22m15.277244178s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680982200 (103h22m15.277753227s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681478081 (103h22m15.27824911s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681895554 (103h22m15.278666585s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682340386 (103h22m15.279111418s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682758716 (103h22m15.279529749s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.683342542 (103h22m15.280113571s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:18.102504696 (103h22m15.699275711s) [upload=115702]: Next returned nil segment
Mar 10 18:22:18.102520582 (103h22m15.699291606s) [upload=115702]: Waiting for error group managing segments...
Mar 10 18:22:18.286488354 (103h22m15.883259367s) [noise=true , address=185.167.204.206:4037 , piece_id=TEYUSWWSWXEKS2TUKAXH7NXD57TQKEZLPS5IOYQVC54ZTY2LEHGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.291859985 (103h22m15.888631012s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.297964517 (103h22m15.894735561s) [noise=true , address=89.149.201.147:4005 , piece_id=CFTY5WE3SZLKOPH6KT4ZD3M2A7ZCIOH7BCUUPGTYM6NRFNJC2BZA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.307699767 (103h22m15.904470781s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.312994778 (103h22m15.909765791s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.319374438 (103h22m15.916145452s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.341507663 (103h22m15.938278696s) [noise=true , address=185.167.204.206:4054 , piece_id=GCJ4ZXR6YMUGI5BG2XOYP4FKTTTSPGVZ5UYS4BOMALGHCSBFQO7Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.351324302 (103h22m15.948095328s) [noise=true , address=81.171.14.70:4032 , piece_id=GPOUM37IH7EL7YANALUZPAQOQ2MHAFOZSVWM7VLDPCNRODOKOO5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.354375553 (103h22m15.951146587s) [noise=true , address=185.167.204.205:4032 , piece_id=YA3TR364FWAMYKQUROV4PYQWTGIXM7SLAUM3BN2FRVBW3MVILHDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.355210000 (103h22m15.951981024s) [noise=true , address=89.149.201.147:4020 , piece_id=AGOZMT6EYQKSFRYOJW4QAV2NMSPM4Q5LAI2SA276EKYK4X5ZGP5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.364878440 (103h22m15.961649464s) [noise=true , address=185.167.204.206:4039 , piece_id=4LSO5WGH5LQ5NIDHOHOACKVAYY37JEZHAUX66MEJ4FIGZ5LWDRFA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.366270355 (103h22m15.963041377s) [noise=true , address=185.167.204.206:4049 , piece_id=JPGUJCUADQNZU5KJAHY4FXUTKJVSEEGMZQ7HNUEWOIWYBJ3LYPEQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.383218910 (103h22m15.979989919s) [noise=true , address=185.167.204.205:4025 , piece_id=2BA453RCULDYAHVTMMVFTPT5MRC4VJNBKK3IHUOLSOMERCHA6LKA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.385734313 (103h22m15.982505326s) [noise=true , address=185.167.204.206:4062 , piece_id=LOWNTJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.386456163 (103h22m15.983227177s) [noise=true , address=185.167.204.206:4071 , piece_id=XT7Y2BSAL4MSA4P6GXDWGQ3R7TGBZT4ON7KAJNVUMBHYPPYU4YTA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389060733 (103h22m15.985831747s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389075564 (103h22m15.985846595s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389787069 (103h22m15.986558092s) [noise=true , address=185.167.204.205:4013 , piece_id=ZYGKKFJQRQZZVEO7DQ67E3ILC66TO5EQBXBAFMKRS2CMKKOKLLVA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.392844303 (103h22m15.989615315s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.395040677 (103h22m15.99181169s) [noise=true , address=185.167.204.206:4053 , piece_id=D72PWJW544HPN6C4C6YIIQBUP7ZIWIN4VCTSYRLGUSCIEUHBWMQA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397224921 (103h22m15.993995937s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397305588 (103h22m15.994076606s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.423243063 (103h22m16.020014089s) [noise=true , address=185.167.204.206:4067 , piece_id=XWH6ZTD6S3TN6TIFN5S4ZIBNR44VOYCB7UCKXOMBBM2MPMNSXUGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:24.985281962 (103h22m22.582052989s) [noise=true , address=185.167.204.205:4007 , piece_id=GDYOPUY5PJFCYLU7OYPBVTW424GT7FP77TMRYKYNKUWXZS6ENAYQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:30.233899651 (103h22m27.830670686s) [noise=true , address=185.167.204.205:4020 , piece_id=5GIMRHZGF2ZEFWLBL4U6KF4DKZ53JZ7PNWB4XTEQW5XCESHNZY5A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:23:17.302478912 (103h23m14.899249945s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection
Mar 10 18:23:17.302510476 (103h23m14.899281492s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 159.223.143.39:39206->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:23:17.302719428 (103h23m14.89949046s) [noise=true , address=185.167.204.206:4058 , piece_id=LZ6PUPONPGDVBLY3IIUUFUZENZTSTWNMCBEK4GVI3Z2RQXICZSDQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: expected piece hash; context deadline exceeded; EOF
Mar 10 18:23:17.302761551 (103h23m14.899532565s) [seg_pos={0 0}, upload=115702]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:23:17.401134721 (103h23m14.997905738s) [upload=115702]: Done uploading segments...
Mar 10 18:23:17.401207170 (103h23m14.997978196s) [upload=115702]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 28.245646615s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [........................................................X..X] duration:
1m0.101781128s trace:
https://jaeger.dev.storj.io/trace/62b857fbddd8c134
FAIL (2026-03-10 18:23:17 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
TJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.678723726 (103h22m15.275494752s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679171303 (103h22m15.275942329s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.679573269 (103h22m15.276344303s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680028932 (103h22m15.27679996s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680473046 (103h22m15.277244178s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.680982200 (103h22m15.277753227s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681478081 (103h22m15.27824911s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.681895554 (103h22m15.278666585s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682340386 (103h22m15.279111418s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.682758716 (103h22m15.279529749s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:17.683342542 (103h22m15.280113571s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Uploading piece...
Mar 10 18:22:18.102504696 (103h22m15.699275711s) [upload=115702]: Next returned nil segment
Mar 10 18:22:18.102520582 (103h22m15.699291606s) [upload=115702]: Waiting for error group managing segments...
Mar 10 18:22:18.286488354 (103h22m15.883259367s) [noise=true , address=185.167.204.206:4037 , piece_id=TEYUSWWSWXEKS2TUKAXH7NXD57TQKEZLPS5IOYQVC54ZTY2LEHGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.291859985 (103h22m15.888631012s) [noise=true , address=81.171.14.70:4031 , piece_id=43ZJLSQWG6ML6VS75ZQ7DFAI56Z7G52P3GXMFKHDQT4ZJKJ333QA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.297964517 (103h22m15.894735561s) [noise=true , address=89.149.201.147:4005 , piece_id=CFTY5WE3SZLKOPH6KT4ZD3M2A7ZCIOH7BCUUPGTYM6NRFNJC2BZA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.307699767 (103h22m15.904470781s) [noise=true , address=89.149.201.147:4010 , piece_id=43BOZGJPSID77Q2EY7ULJQIK45WZ4IEZZ4HTFQ43J7TP2SIXJQKQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.312994778 (103h22m15.909765791s) [noise=true , address=185.167.204.205:4017 , piece_id=GLOJ3UR3HQZRVVZ3BMUAXSQBK6YJ4MX5EBGPH6LNET7JDH4RWEXQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.319374438 (103h22m15.916145452s) [noise=true , address=81.171.14.70:4048 , piece_id=JYSM26GYUVQK4QTC444MOC4KXPWBU5OUVH5QPWQGP72AGFBC2MIQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.341507663 (103h22m15.938278696s) [noise=true , address=185.167.204.206:4054 , piece_id=GCJ4ZXR6YMUGI5BG2XOYP4FKTTTSPGVZ5UYS4BOMALGHCSBFQO7Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.351324302 (103h22m15.948095328s) [noise=true , address=81.171.14.70:4032 , piece_id=GPOUM37IH7EL7YANALUZPAQOQ2MHAFOZSVWM7VLDPCNRODOKOO5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.354375553 (103h22m15.951146587s) [noise=true , address=185.167.204.205:4032 , piece_id=YA3TR364FWAMYKQUROV4PYQWTGIXM7SLAUM3BN2FRVBW3MVILHDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.355210000 (103h22m15.951981024s) [noise=true , address=89.149.201.147:4020 , piece_id=AGOZMT6EYQKSFRYOJW4QAV2NMSPM4Q5LAI2SA276EKYK4X5ZGP5Q, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.364878440 (103h22m15.961649464s) [noise=true , address=185.167.204.206:4039 , piece_id=4LSO5WGH5LQ5NIDHOHOACKVAYY37JEZHAUX66MEJ4FIGZ5LWDRFA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.366270355 (103h22m15.963041377s) [noise=true , address=185.167.204.206:4049 , piece_id=JPGUJCUADQNZU5KJAHY4FXUTKJVSEEGMZQ7HNUEWOIWYBJ3LYPEQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.383218910 (103h22m15.979989919s) [noise=true , address=185.167.204.205:4025 , piece_id=2BA453RCULDYAHVTMMVFTPT5MRC4VJNBKK3IHUOLSOMERCHA6LKA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.385734313 (103h22m15.982505326s) [noise=true , address=185.167.204.206:4062 , piece_id=LOWNTJEFWAHKTTCYX2TMYQOT5XKRZQQAUWT7XHPFBHBHLDJFYQQQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.386456163 (103h22m15.983227177s) [noise=true , address=185.167.204.206:4071 , piece_id=XT7Y2BSAL4MSA4P6GXDWGQ3R7TGBZT4ON7KAJNVUMBHYPPYU4YTA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389060733 (103h22m15.985831747s) [noise=true , address=185.167.204.205:4001 , piece_id=5T7AKGPWFTSCKDMKK3BOLGSIH3OAVNAZB3D5ILXXSYOR2CA2NFHQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389075564 (103h22m15.985846595s) [noise=true , address=185.167.204.206:4068 , piece_id=ILUQHWCZH6TERJONKPNEQCZKFZF7TD757AGIQU3DD7HWSKQ52NDA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.389787069 (103h22m15.986558092s) [noise=true , address=185.167.204.205:4013 , piece_id=ZYGKKFJQRQZZVEO7DQ67E3ILC66TO5EQBXBAFMKRS2CMKKOKLLVA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.392844303 (103h22m15.989615315s) [noise=true , address=185.167.204.205:4030 , piece_id=OYNJ3OVLH2SXV4I432BJOXP3YPCAOML62OTFNYBFMN47PQ5UGWAA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.395040677 (103h22m15.99181169s) [noise=true , address=185.167.204.206:4053 , piece_id=D72PWJW544HPN6C4C6YIIQBUP7ZIWIN4VCTSYRLGUSCIEUHBWMQA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397224921 (103h22m15.993995937s) [noise=true , address=185.167.204.205:4036 , piece_id=WZ5MCIX5JEJ5QNDULLGIVXNKQBVFEX7ATOQNBTETSOW2WCNWYSJQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.397305588 (103h22m15.994076606s) [noise=true , address=89.149.201.147:4016 , piece_id=ELN7QQHMWZICYFEWCRGUR5OUEVLKDNKHK5Y3KEQXDFRCE3SWEB2A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:18.423243063 (103h22m16.020014089s) [noise=true , address=185.167.204.206:4067 , piece_id=XWH6ZTD6S3TN6TIFN5S4ZIBNR44VOYCB7UCKXOMBBM2MPMNSXUGA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:24.985281962 (103h22m22.582052989s) [noise=true , address=185.167.204.205:4007 , piece_id=GDYOPUY5PJFCYLU7OYPBVTW424GT7FP77TMRYKYNKUWXZS6ENAYQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:22:30.233899651 (103h22m27.830670686s) [noise=true , address=185.167.204.205:4020 , piece_id=5GIMRHZGF2ZEFWLBL4U6KF4DKZ53JZ7PNWB4XTEQW5XCESHNZY5A, seg_pos={0 0}, upload=115702]: Done uploading piece. err: <nil>
Mar 10 18:23:17.302478912 (103h23m14.899249945s) [noise=true , address=185.167.204.205:4024 , piece_id=MCKCFKQPVK4H7YWVNQBYD7VLK26D4ABODUMWYEESU22S2GZ2X4UA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:57608->185.167.204.205:4024: use of closed network connection
Mar 10 18:23:17.302510476 (103h23m14.899281492s) [noise=true , address=89.149.201.147:4014 , piece_id=VZUX5U4EDER24ANUL7OGMVS5JWIE45RT5HINRYU662ULMKG3Q6SA, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 159.223.143.39:39206->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:23:17.302719428 (103h23m14.89949046s) [noise=true , address=185.167.204.206:4058 , piece_id=LZ6PUPONPGDVBLY3IIUUFUZENZTSTWNMCBEK4GVI3Z2RQXICZSDQ, seg_pos={0 0}, upload=115702]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: expected piece hash; context deadline exceeded; EOF
Mar 10 18:23:17.302761551 (103h23m14.899532565s) [seg_pos={0 0}, upload=115702]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:23:17.401134721 (103h23m14.997905738s) [upload=115702]: Done uploading segments...
Mar 10 18:23:17.401207170 (103h23m14.997978196s) [upload=115702]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 1m0.101805813s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
2.081260796s trace:
https://jaeger.dev.storj.io/trace/5e95e001955ee5b9
FAIL (2026-03-10 18:21:31 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
, address=89.149.201.147:4014 , piece_id=UQSDYL6TIICACSW3B64V3BAZWEOUVTBQMK44HRFWYQTOX2JSTPJQ, seg_pos={0 0}, upload=115653]: Uploading piece...
Mar 10 18:19:35.226415607 (103h19m32.823186625s) [noise=true , address=185.167.204.206:4049 , piece_id=HD5TQPI6NW6GZLA2GHM5CLGKTOM4LYIQYH6D4BZ5EMD52YZDBY3Q, seg_pos={0 0}, upload=115653]: Uploading piece...
Mar 10 18:19:35.229137811 (103h19m32.825908827s) [noise=true , address=89.149.201.147:4016 , piece_id=HUUUQVWZSFDIHHMGZRCD3K6GK57SQKDBO7HMQ6DBO4Y5FHK4E3AA, seg_pos={0 0}, upload=115653]: Uploading piece...
Mar 10 18:19:35.232909424 (103h19m32.829680437s) [noise=true , address=185.167.204.205:4020 , piece_id=5FJQI37K5R74ZUWXCJZWKFBKKZSXPPZN4M4HKJ25E5FK7XWQY3BQ, seg_pos={0 0}, upload=115653]: Uploading piece...
Mar 10 18:19:35.239159038 (103h19m32.835930054s) [noise=true , address=185.167.204.205:4005 , piece_id=7Y6XW2ZDGPSKPSJ3JEOKAFRRG5PUSGXTN6GQDN5TUV5X4ZCZ2WXQ, seg_pos={0 0}, upload=115653]: Uploading piece...
Mar 10 18:19:35.241027123 (103h19m32.837798137s) [noise=true , address=89.149.201.147:4009 , piece_id=ECNFO4GOL5NSXUNS6DBSELF3623KUNPTQXUDGDU2CNBEUHMJXZ4Q, seg_pos={0 0}, upload=115653]: Uploading piece...
Mar 10 18:19:35.242045917 (103h19m32.838816934s) [noise=true , address=185.167.204.206:4050 , piece_id=SO6FICYXPSKGQLHNDY4Q4HMSS6HIZHCGFSXZTR4ZVV73FFTYELUQ, seg_pos={0 0}, upload=115653]: Uploading piece...
Mar 10 18:19:35.244248381 (103h19m32.84101939s) [noise=true , address=185.167.204.205:4014 , piece_id=RC4G6B36AJ5RAZ4AB45OHJKWEWHVGCVBN2KBOGOGMIF4BEYOO3OQ, seg_pos={0 0}, upload=115653]: Uploading piece...
Mar 10 18:19:35.246388125 (103h19m32.843159141s) [noise=true , address=81.171.14.70:4035 , piece_id=WB5O2MZ4VZRDWX6TB5FP3XFO4INBMNZM7DTGIXHXF6WJ3Z5BY6EQ, seg_pos={0 0}, upload=115653]: Uploading piece...
Mar 10 18:19:35.246525590 (103h19m32.843296608s) [noise=true , address=185.167.204.206:4043 , piece_id=I2GTD7RZFP5LBOLG24KFOMZRR6GRLB3VXPJUT7KC2S7ADPOPUDRA, seg_pos={0 0}, upload=115653]: Uploading piece...
Mar 10 18:19:35.248403422 (103h19m32.845174441s) [noise=true , address=89.149.201.147:4002 , piece_id=UB3WE3LIHTK7MC6I5L6WJBDFB3EMTVVL6ZMQWEQWLUHEKDGV4XFQ, seg_pos={0 0}, upload=115653]: Uploading piece...
Mar 10 18:19:35.248951823 (103h19m32.84572284s) [noise=true , address=185.167.204.205:4019 , piece_id=XVPVNOTQTPJPRGXFH67TVVXYCEDC7U6XP2RPA4KWTX4AVBFWUXEQ, seg_pos={0 0}, upload=115653]: Uploading piece...
Mar 10 18:19:35.616832511 (103h19m33.213603542s) [upload=115653]: Next returned nil segment
Mar 10 18:19:35.616849321 (103h19m33.213620369s) [upload=115653]: Waiting for error group managing segments...
Mar 10 18:19:35.810806844 (103h19m33.407577867s) [noise=true , address=81.171.14.70:4043 , piece_id=KG3YSFUEWQUPUL5HPMUZVUYE6MYPK2GVQQFIGO4IWUXVRMSAVSHA, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.810960076 (103h19m33.407731099s) [noise=true , address=81.171.14.70:4030 , piece_id=N3C7SQ4Y26RJVC5EAB5ASEMEMPDO2YAT4X6R2FGGSQHCMTOUME7A, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.818910271 (103h19m33.415681302s) [noise=true , address=81.171.14.70:4047 , piece_id=NTUAA6WV4DIVMMYOTSJWZP4PEDA2STSMGXDEIWJOEG66GV2IP52A, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.821210624 (103h19m33.417981649s) [noise=true , address=185.167.204.206:4044 , piece_id=IOGPHO2IMLM4WJPXRYEODPAILYVAW35E7OYNBATYMOADMHBVTAEA, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.834331969 (103h19m33.431102982s) [noise=true , address=81.171.14.70:4025 , piece_id=NWFY7S6Z4P2JZRKNB7DCJHH6KVWEJX673OYQAYAAQYBOOBQMN7KA, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.867592710 (103h19m33.464363736s) [noise=true , address=89.149.201.147:4016 , piece_id=HUUUQVWZSFDIHHMGZRCD3K6GK57SQKDBO7HMQ6DBO4Y5FHK4E3AA, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.869995030 (103h19m33.466766053s) [noise=true , address=185.167.204.205:4007 , piece_id=VZDOYSXJNSUBIBFOWPFSA2P5K4MI2QYJG4RQ4TQ2TJHFR6NMY5HA, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.874770892 (103h19m33.471541929s) [noise=true , address=185.167.204.205:4016 , piece_id=JUTDYPCLBRXBPXROVUIXF46FQ5R6CMFVX6NWYNTNBA4ILORZPJVQ, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.889847511 (103h19m33.486618544s) [noise=true , address=89.149.201.147:4006 , piece_id=URPQHEVCEFNWMVX2UDBUOEXNGCIA7CNN6CTRBEQ4QV3FX3PUFOSA, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.894492116 (103h19m33.491263146s) [noise=true , address=89.149.201.147:4014 , piece_id=UQSDYL6TIICACSW3B64V3BAZWEOUVTBQMK44HRFWYQTOX2JSTPJQ, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.899603360 (103h19m33.496374369s) [noise=true , address=89.149.201.147:4022 , piece_id=XX32YI3C5N75XNYAAFPGO4J76KWB2KAVR7OYXFBUYL45IAZP5QNQ, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.909066435 (103h19m33.505837459s) [noise=true , address=185.167.204.206:4070 , piece_id=BO4U37EMJ3FW5JILWTXBCZDEJWJLNSXC2IVTDXIMBILLVT4P2KYQ, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.909655371 (103h19m33.506426399s) [noise=true , address=89.149.201.147:4015 , piece_id=7OJ7MOPW7UG3Y7VQVM7Y5BWHYAJFEPTOQPBAPODLQBGAJYU7ERTQ, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.912670765 (103h19m33.509441793s) [noise=true , address=185.167.204.205:4005 , piece_id=7Y6XW2ZDGPSKPSJ3JEOKAFRRG5PUSGXTN6GQDN5TUV5X4ZCZ2WXQ, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.917769965 (103h19m33.514540992s) [noise=true , address=185.167.204.205:4014 , piece_id=RC4G6B36AJ5RAZ4AB45OHJKWEWHVGCVBN2KBOGOGMIF4BEYOO3OQ, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.925026294 (103h19m33.521797319s) [noise=true , address=185.167.204.206:4050 , piece_id=SO6FICYXPSKGQLHNDY4Q4HMSS6HIZHCGFSXZTR4ZVV73FFTYELUQ, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.931218169 (103h19m33.527989196s) [noise=true , address=185.167.204.205:4020 , piece_id=5FJQI37K5R74ZUWXCJZWKFBKKZSXPPZN4M4HKJ25E5FK7XWQY3BQ, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.932952746 (103h19m33.529723781s) [noise=true , address=185.167.204.205:4009 , piece_id=XN73S4H5BGQKLUQWICEXRH7RQV4L7FYGS6CFDBH3V53NGBZBDAMQ, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.944198414 (103h19m33.540969446s) [noise=true , address=185.167.204.205:4019 , piece_id=XVPVNOTQTPJPRGXFH67TVVXYCEDC7U6XP2RPA4KWTX4AVBFWUXEQ, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.950937978 (103h19m33.547708991s) [noise=true , address=185.167.204.205:4035 , piece_id=H4OKKVGRHJZCTASRNNETURODJE7FC2DUZJD6CKD3ONEWNY36PGNA, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.951080419 (103h19m33.547851432s) [noise=true , address=185.167.204.205:4028 , piece_id=TC57OJQ6BA24NO27OGHI7KBVPCJK35IV5EQ6SQAKQL46BA3YWDEA, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:35.956570986 (103h19m33.553342013s) [noise=true , address=185.167.204.206:4049 , piece_id=HD5TQPI6NW6GZLA2GHM5CLGKTOM4LYIQYH6D4BZ5EMD52YZDBY3Q, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:36.260422274 (103h19m33.857193301s) [noise=true , address=89.149.201.147:4010 , piece_id=XOB2KBJX2JMBITMAKDGHQKSTX3H63WRW2WWPMXI5RD5ZP7F7FBLA, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:36.294530007 (103h19m33.89130103s) [noise=true , address=81.171.14.70:4035 , piece_id=WB5O2MZ4VZRDWX6TB5FP3XFO4INBMNZM7DTGIXHXF6WJ3Z5BY6EQ, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:19:36.340503759 (103h19m33.937274801s) [noise=true , address=89.149.201.147:4020 , piece_id=I2KYV66QPQRWS3C27NKWQDW4ITYSLFB6XUCY7UHZLQK5WKYTAU4Q, seg_pos={0 0}, upload=115653]: Done uploading piece. err: <nil>
Mar 10 18:20:34.740126747 (103h20m32.336897775s) [noise=true , address=89.149.201.147:4002 , piece_id=UB3WE3LIHTK7MC6I5L6WJBDFB3EMTVVL6ZMQWEQWLUHEKDGV4XFQ, seg_pos={0 0}, upload=115653]: Done uploading piece. err: ecclient: upload failed (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc, address:89.149.201.147:4002): protocol: read tcp 159.223.143.39:33340->89.149.201.147:4002: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:20:34.740414606 (103h20m32.337185632s) [noise=true , address=185.167.204.206:4043 , piece_id=I2GTD7RZFP5LBOLG24KFOMZRR6GRLB3VXPJUT7KC2S7ADPOPUDRA, seg_pos={0 0}, upload=115653]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 159.223.143.39:56250->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:20:34.740666088 (103h20m32.337437119s) [noise=true , address=89.149.201.147:4009 , piece_id=ECNFO4GOL5NSXUNS6DBSELF3623KUNPTQXUDGDU2CNBEUHMJXZ4Q, seg_pos={0 0}, upload=115653]: Done uploading piece. err: ecclient: upload failed (node:1sbZWJFMVsppCpbFVfn33bx5uGkhtFpzJm8a2i8DoAfQNsWY5F, address:89.149.201.147:4009): protocol: read tcp 159.223.143.39:34182->89.149.201.147:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:20:34.740735449 (103h20m32.337506487s) [seg_pos={0 0}, upload=115653]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Mar 10 18:20:34.837288406 (103h20m32.434059429s) [upload=115653]: Done uploading segments...
Mar 10 18:20:34.837328146 (103h20m32.434099169s) [upload=115653]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.081282865s
core
uplink
eu-select-1
eu1-placement-15
|