Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 1.266289844s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:24:02 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.266327781s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 1.133864323s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:22:58 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.134205741s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 947.843419ms trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:22:25 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 947.88882ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 1.118838403s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:21:59 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.118887321s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 1.261847634s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:21:06 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.261875726s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 708.847727ms trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:20:18 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 708.878972ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 1.30173546s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:19:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.301766832s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 1.279149385s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:18:58 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.279238269s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 1.287370274s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:18:43 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.287403122s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 1.322353808s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:18:07 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.322390801s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 1.267521977s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:17:16 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.267554451s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 1.180920827s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:16:29 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.180948132s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 1.331724658s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:15:27 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.331758814s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 1.26805979s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:14:12 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.268091759s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 960.367899ms trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:13:53 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 960.400557ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 1.251502641s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:13:30 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.25156667s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 1.018401086s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:12:38 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.018429817s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 1.288345093s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:12:07 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.288377646s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.285137148s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:10:54 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.285164977s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 1.306001562s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:09:55 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.306140174s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 828.773441ms trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:09:16 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 828.801553ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 1.308320989s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:08:14 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.308352872s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 1.358212444s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:07:06 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.358246202s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 1.180037298s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:06:39 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.180066329s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 1.295023077s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:06:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.295090957s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 1.301028041s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:05:09 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.301057111s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 1.170005345s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:04:32 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.170047952s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 1.271763719s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:03:49 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.271807353s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 1.2000969s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:03:06 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.200112763s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 1.185541463s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:02:33 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.185572552s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 754.129774ms trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:01:28 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 754.161983ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 1.315172501s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:01:00 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.315212334s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 982.197959ms trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:00:28 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 982.227815ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 1.236440638s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 17:00:06 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.236473106s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 1.262434667s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 16:59:41 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.262462691s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 1.157286983s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 16:58:59 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.157305785s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 1.2706904s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 16:58:26 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.27072s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 891.107138ms trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 16:57:45 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 891.159757ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 1.111798045s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 16:57:29 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.111829788s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 1.237024391s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 16:56:41 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.237057009s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 1.137907848s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 16:55:27 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.137936216s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 1.251351487s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 16:55:10 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.251461659s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 1.111369035s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 16:54:49 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.111395905s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 1.288772682s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 16:54:23 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.288800321s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 735.704842ms trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 16:53:29 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 735.763055ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 1.253222322s trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 16:52:24 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 1.253253045s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 756.493335ms trace: https://jaeger.dev.storj.io/trace/72d1768dadcbe46b FAIL (2026-03-18 16:51:10 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: internal error ======LOGS====== 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.626765311 (293h51m7.22353634s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.627040986 (293h51m7.22381201s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628025201 (293h51m7.224796227s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628325911 (293h51m7.225096946s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628627099 (293h51m7.225398122s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.628839133 (293h51m7.225610159s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629107755 (293h51m7.225878781s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629262958 (293h51m7.226033985s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629635295 (293h51m7.226406325s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.629653133 (293h51m7.226424146s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630058646 (293h51m7.226829659s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630150638 (293h51m7.226921661s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630482158 (293h51m7.227253192s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630583078 (293h51m7.227354102s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630979980 (293h51m7.227751009s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.630991686 (293h51m7.22776271s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.631478670 (293h51m7.228249705s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Uploading piece... Mar 18 16:51:09.702440166 (293h51m7.299211182s) [upload=329111]: Next returned nil segment Mar 18 16:51:09.702456559 (293h51m7.299227582s) [upload=329111]: Waiting for error group managing segments... Mar 18 16:51:09.708407591 (293h51m7.305178602s) [noise=true , address=45.91.116.14:4223 , piece_id=Y2LJF2T2JF6JEB3XEOUEKW7XXJSZFA7GIW7S4YE4OZGJZY2T4GAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.712495525 (293h51m7.309266553s) [noise=true , address=145.79.227.34:4929 , piece_id=4IIPYX7673RIL67LZL4SSHOKOE6ZPHVLULEMO2KUIM6JA35QGAAQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.725343753 (293h51m7.322114771s) [noise=true , address=45.91.116.11:4143 , piece_id=ELVNAOVV34L343UP5JIUG6WIWMLF45WIHLM35YN5QQIZQDKHH2ZA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.726981573 (293h51m7.323752582s) [noise=true , address=45.91.116.7:4039 , piece_id=6HJFURMEX4XXUT4IBEJ7ECIVGJDSGFSV632BDJFWY454ZFEDVPUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.727988660 (293h51m7.324759695s) [noise=true , address=145.79.227.18:4027 , piece_id=Z5PLBA54QIJPPLZAXQ5DAPI5ML37IVQ4FWZHFENB5GHA6GPPG7LQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.733311274 (293h51m7.330082286s) [noise=true , address=79.127.219.46:4903 , piece_id=UPMVRM62GELNI7LQZJD3GIEVPNAM2WDTRFO3HCHCSB2TXFS3VWSQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.739872034 (293h51m7.336643065s) [noise=true , address=77.90.31.18:4836 , piece_id=L3AVOEUDR6SXRVUFXPGVQKO7P2HZGQIGLXP3BQFREEHZIGMOBIUA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.745323402 (293h51m7.342094431s) [noise=true , address=79.127.219.38:4848 , piece_id=SOT23DPU4PZFPHRRSBGNCVAREMFW7NXSILPKN4A7TBUTZYTQSVQA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.762176142 (293h51m7.358947165s) [noise=true , address=45.91.116.10:4086 , piece_id=HFCEUWSSPKMNOALZCLCYQFIGLGRXYWTJ2SL5S7EK4OQFOVMOEFJA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.959127349 (293h51m7.55589838s) [noise=true , address=167.17.72.70:4778 , piece_id=JGWVQAXS3WC4DLJ3FTDRCO3TTXEUQBLBKEV4OLQWZBVT2H7KZXNQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.974703744 (293h51m7.571474768s) [noise=true , address=167.17.72.69:4710 , piece_id=QYEKCEVYXUOJNTCPB45SWVAZDXH4PFNBGDHFTUH6M5CLEZMQ7NOQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:09.984676068 (293h51m7.581447083s) [noise=true , address=167.17.72.68:4669 , piece_id=KMC3K46EOM7PHFCKPJZAPV3PCENB7M5R6JWVRZJ7O4OAZDJQVEQQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.186312524 (293h51m7.78308355s) [noise=true , address=79.127.205.237:4222 , piece_id=GNXIJOMWYHSGMDASN5FWBLVHNVMG3NB26UVROISLP7VKI2VI7FEQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209670436 (293h51m7.806441459s) [noise=true , address=79.127.205.233:4162 , piece_id=QSD4X5KFQIV5UHI223H4WW5WGOXKBCAIVX3MLYZFGBLAMSB6SZ6Q, seg_pos={0 0}, upload=329111]: Done uploading piece. err: <nil> Mar 18 16:51:10.209695349 (293h51m7.806466373s) [seg_pos={0 0}, upload=329111]: Satisfied cohort requirements after 14 pieces Mar 18 16:51:10.209975349 (293h51m7.806746384s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BhogobSTS5pCJxh7NGgnsU5qn1kwDDv25XxQiqqPXGSU9QL9H): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.209988536 (293h51m7.80675957s) [noise=true , address=193.149.164.67:4020 , piece_id=REOKCOPHYMKUIVKBPCGOBNASQ35YX55EMYTDCMOOSDTROOQIHAIQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210087026 (293h51m7.806858058s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qcwvrgkkT6Q8pVEuUu7fA4wwHXNidvCNnKhrU3DpdqAHKGtLG): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210112972 (293h51m7.806883998s) [noise=true , address=193.149.164.69:4077 , piece_id=HLBUOFEG2KHIU3YDA4VFK45MNJ7SQ4H3ASOOZVQOBMTQRSILAWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210410740 (293h51m7.807181771s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cHb8XLk563sSJ6s92CpzfmxwjiydGAe3GjZWqEsZgcmzRqpLR): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210422131 (293h51m7.807193165s) [noise=true , address=193.149.164.68:4047 , piece_id=KI2GOOUSR4RXRBDRUPDFTPC7H5X7EARHWMMFHBV4YGRUOOZXWWUQ, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210448494 (293h51m7.807219527s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M7G5jzPdqBzSjhVDsktUNK4MDhsBZ9b4BexkjXyVrGCXaujE1): protocol: expected piece hash; context canceled; EOF Mar 18 16:51:10.210461044 (293h51m7.80723208s) [noise=true , address=193.149.164.70:4119 , piece_id=NNFYTOCFKSJ3SAGTF5CKFPV5ZQOKTZOOLMNZG55ZW3MS7JWESEIA, seg_pos={0 0}, upload=329111]: Long tail cancellation due to optimal threshold Mar 18 16:51:10.210506936 (293h51m7.807277951s) [seg_pos={0 0}, upload=329111]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 18 16:51:10.210602806 (293h51m7.80737383s) [upload=329111]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 16:51:10.210609332 (293h51m7.807380355s) [upload=329111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 16:51:10.262661082 (293h51m7.859432097s) [upload=329111]: Done uploading segments... Mar 18 16:51:10.262684233 (293h51m7.859455246s) [upload=329111]: Upload finished. err: <nil> Duration: 756.519761ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.967693246s trace: https://jaeger.dev.storj.io/trace/5b11c5dc7ef8664 FAIL (2026-03-18 16:08:51 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 1.967723781s core uplink uk-select-1 uk1-placement-17
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.900449659s trace: https://jaeger.dev.storj.io/trace/5b11c5dc7ef8664 FAIL (2026-03-18 16:08:27 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 1.900626013s core uplink uk-select-1 uk1-placement-17
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 2 out of 60 is failed (threshold=3) [X.X.........................................................] duration: 1.903266617s trace: https://jaeger.dev.storj.io/trace/5b11c5dc7ef8664 FAIL (2026-03-18 16:07:58 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 1.903298243s core uplink uk-select-1 uk1-placement-17

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)