Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 486.957527ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:30:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 487.046357ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 472.361613ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:29:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 472.39752ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 478.004627ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:28:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 478.100062ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 411.935669ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:27:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 411.965627ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 549.638851ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:26:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 549.666355ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 559.780595ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:25:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 559.809957ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 438.635117ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:24:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 438.668608ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 553.193829ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:24:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 553.228048ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 452.475021ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:23:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 452.505803ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 432.572119ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:22:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 432.603517ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 511.296069ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:21:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 511.330876ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 539.905664ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:20:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 539.986932ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 442.128997ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:19:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 442.159077ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 580.010776ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:19:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 580.240023ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 475.02737ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:18:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 475.059468ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 411.57524ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:17:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 411.603968ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 471.711097ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:17:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 471.741508ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 465.10247ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:16:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 465.136116ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 492.118628ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:15:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 492.146746ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 598.071498ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:14:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 598.106555ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 378.738129ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:13:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 378.802128ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 436.993451ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:13:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 437.02204ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 468.40612ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:11:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 468.435957ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 439.276185ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:11:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 439.304938ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 474.312913ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:11:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 474.407365ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 611.948426ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:10:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 611.982539ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.262825949s trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:09:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.262885084s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 517.099439ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:07:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 517.130346ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 406.27843ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:07:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 406.31053ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 519.944348ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:06:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 520.082523ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 460.011204ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:06:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 460.068806ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 439.943132ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:05:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 439.966088ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 470.165325ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:05:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 470.193849ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 436.681698ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:04:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 436.713581ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 426.182831ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:03:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 426.216221ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 507.455173ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:02:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 507.488451ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 424.315862ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:01:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 424.352573ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 418.381409ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:00:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 418.436862ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 601.676466ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 20:00:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 601.720547ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 457.20038ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 19:59:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 457.228982ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 519.07661ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 19:58:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 519.119498ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 449.278782ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 19:57:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 449.31893ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 444.988951ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 19:57:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 445.022383ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 587.00486ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 19:55:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 587.041768ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 435.366246ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 19:55:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 435.433661ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 442.766445ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 19:54:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 442.819217ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 570.277616ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 19:53:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 570.308753ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 466.161218ms trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 19:53:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 466.194458ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.036888865s trace: https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c FAIL (2026-03-17 19:53:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments... Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment... Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment. Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece... Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments... Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil> Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments... Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.036939127s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.571522582s trace: https://jaeger.dev.storj.io/trace/32e71cb3854943d3 FAIL (2026-03-17 19:27:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== true , address=185.167.204.206:4059 , piece_id=BYFO252V4VQSTUR5ZQH5Z5GPZ5UWQUNYYM5KEMOQD7T4QTWXO3RQ, seg_pos={0 0}, upload=304355]: Uploading piece... Mar 17 18:43:50.596939273 (271h43m48.193710296s) [noise=true , address=185.167.204.206:4062 , piece_id=GKR5RHHPTGRGBAIAA45KHX4Z4WP7IPLI5K6SOU56CKRGPB2UXLAA, seg_pos={0 0}, upload=304355]: Uploading piece... Mar 17 18:43:50.597598355 (271h43m48.194369387s) [noise=true , address=81.171.14.70:4035 , piece_id=EU3TS6VRI3UEG7DLRDFC6FQ6M24R3CGMUC5JU2Q6NRC4VRLOKYHQ, seg_pos={0 0}, upload=304355]: Uploading piece... Mar 17 18:43:50.598635003 (271h43m48.195406029s) [noise=true , address=81.171.14.70:4039 , piece_id=TOC7KZHNZQAEURL4STHUX4O5NYCY3UALRLTREU4AZTXQOJV3XCAA, seg_pos={0 0}, upload=304355]: Uploading piece... Mar 17 18:43:50.599426619 (271h43m48.196197652s) [noise=true , address=185.167.204.206:4052 , piece_id=LTEFUECRJ7FP4LKRJOIOTTLNLQTG2VSY46QLJO5TISSPXD4STPCA, seg_pos={0 0}, upload=304355]: Uploading piece... Mar 17 18:43:50.601148281 (271h43m48.197919313s) [noise=true , address=185.167.204.206:4050 , piece_id=2HVVFROYA4GVKMDG7RA6JB5SVB752NDMS6ST2AH7A7C7OCOPTYOA, seg_pos={0 0}, upload=304355]: Uploading piece... Mar 17 18:43:50.604112007 (271h43m48.200883038s) [noise=true , address=81.171.14.70:4025 , piece_id=BA7DAZT6RECHH67PJOOOD6K7WSKFSEHGJUJPAK3Y7EJHFAJTLQHQ, seg_pos={0 0}, upload=304355]: Uploading piece... Mar 17 18:43:50.606640883 (271h43m48.203411916s) [noise=true , address=89.149.201.147:4004 , piece_id=7MU6DJLPBPAYUQNXVHNJ2VIXQ5XC4FGNACJVBLLFLYMBXR7VADHQ, seg_pos={0 0}, upload=304355]: Uploading piece... Mar 17 18:43:50.609000949 (271h43m48.205771983s) [noise=true , address=185.167.204.205:4026 , piece_id=IINFYHULKK3NVITBYGTGGZSTGVS5IHD4Y3PYJ2ZI44U4GJHYKEVQ, seg_pos={0 0}, upload=304355]: Uploading piece... Mar 17 18:43:50.613266292 (271h43m48.210037324s) [noise=true , address=81.171.14.70:4028 , piece_id=T5HQEGZZVW3H7N2PHWUSZK4Y6CGGV7H7LHMLKUB44JK7ITODJ44A, seg_pos={0 0}, upload=304355]: Uploading piece... Mar 17 18:43:50.617071393 (271h43m48.213842425s) [noise=true , address=81.171.14.70:4033 , piece_id=PPK355IMSA264EKUH4XRA3EMSRW3ZAPROZZOYBWBWYPGKS6FDXSA, seg_pos={0 0}, upload=304355]: Uploading piece... Mar 17 18:43:50.620780797 (271h43m48.21755183s) [noise=true , address=81.171.14.70:4047 , piece_id=VNO3Y4EGCW7JHR3NYKJG5PSRBLLZBZLH6TJN6WUITYNLNLLVVSFA, seg_pos={0 0}, upload=304355]: Uploading piece... Mar 17 18:43:51.014144493 (271h43m48.610915526s) [upload=304355]: Next returned nil segment Mar 17 18:43:51.014161653 (271h43m48.610932688s) [upload=304355]: Waiting for error group managing segments... Mar 17 18:43:51.208306536 (271h43m48.805077566s) [noise=true , address=81.171.14.70:4027 , piece_id=YSHNTODSDIKZESC5RJBXFVMK6P5XTS7RTOWYDBS2O7FXAX7T2AFQ, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.213317344 (271h43m48.810088383s) [noise=true , address=81.171.14.70:4040 , piece_id=YNEG56UE7FT43CXKANNJSOSG3GVYHJ4FG2V3PQI6JW5T5UXM2V6Q, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.223526049 (271h43m48.820297087s) [noise=true , address=81.171.14.70:4036 , piece_id=2XVM5F7MXMBVLYB5YAX3TILEUMXSPDOZWGDQDOOYGXBGSLCNLXSA, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.230237682 (271h43m48.827008698s) [noise=true , address=81.171.14.70:4035 , piece_id=EU3TS6VRI3UEG7DLRDFC6FQ6M24R3CGMUC5JU2Q6NRC4VRLOKYHQ, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.240039073 (271h43m48.836810101s) [noise=true , address=81.171.14.70:4033 , piece_id=PPK355IMSA264EKUH4XRA3EMSRW3ZAPROZZOYBWBWYPGKS6FDXSA, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.241636501 (271h43m48.83840753s) [noise=true , address=81.171.14.70:4045 , piece_id=NWBMHS5W7FU3C2TGI3EVE6XLVSUN2NMYMU6IZEZ2EXULJWG2INUA, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.244747460 (271h43m48.841518486s) [noise=true , address=81.171.14.70:4029 , piece_id=KDXXMUC24MJP4X3NVA5W5365AQFER5T2OYYUCUD7YHY566OVHKKA, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.249283839 (271h43m48.84605487s) [noise=true , address=89.149.201.147:4023 , piece_id=E6P2KN7EMYBPI7ANF6I7RW4SVMGDSIIPHZF2TEIIFPMXVQEQMVFA, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.250696479 (271h43m48.847467504s) [noise=true , address=81.171.14.70:4028 , piece_id=T5HQEGZZVW3H7N2PHWUSZK4Y6CGGV7H7LHMLKUB44JK7ITODJ44A, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.254491465 (271h43m48.85126249s) [noise=true , address=81.171.14.70:4039 , piece_id=TOC7KZHNZQAEURL4STHUX4O5NYCY3UALRLTREU4AZTXQOJV3XCAA, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.256138328 (271h43m48.852909374s) [noise=true , address=89.149.201.147:4004 , piece_id=7MU6DJLPBPAYUQNXVHNJ2VIXQ5XC4FGNACJVBLLFLYMBXR7VADHQ, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.257247887 (271h43m48.854018918s) [noise=true , address=81.171.14.70:4042 , piece_id=RSPYM5SBTRQVL676SWCNCPUZP7WVZNPCTGIOBFQYC7YNBQXBQRJQ, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.264714686 (271h43m48.861485717s) [noise=true , address=81.171.14.70:4043 , piece_id=5MHRFDUDVEH7QFXPUTOPADY67ARND3TEQZXV5IGOYQYVB4OP22ZQ, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.267069109 (271h43m48.863840133s) [noise=true , address=81.171.14.70:4047 , piece_id=VNO3Y4EGCW7JHR3NYKJG5PSRBLLZBZLH6TJN6WUITYNLNLLVVSFA, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.285081421 (271h43m48.881852451s) [noise=true , address=81.171.14.70:4025 , piece_id=BA7DAZT6RECHH67PJOOOD6K7WSKFSEHGJUJPAK3Y7EJHFAJTLQHQ, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.286012931 (271h43m48.882783949s) [noise=true , address=185.167.204.206:4052 , piece_id=LTEFUECRJ7FP4LKRJOIOTTLNLQTG2VSY46QLJO5TISSPXD4STPCA, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.291987259 (271h43m48.888758287s) [noise=true , address=89.149.201.147:4001 , piece_id=37WCZ2UPC7YZCDBHCLQPLDLNMCKX53SA66SICWYUZDGZNEYH5P4Q, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.322595472 (271h43m48.919366497s) [noise=true , address=89.149.201.147:4002 , piece_id=MCG72U46VSRYIY2YCVRNJCVFTPPX5VH6J7UWDLXN4UTMHP6Y4A6Q, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.340454625 (271h43m48.937225639s) [noise=true , address=185.167.204.206:4062 , piece_id=GKR5RHHPTGRGBAIAA45KHX4Z4WP7IPLI5K6SOU56CKRGPB2UXLAA, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.340718566 (271h43m48.937489593s) [noise=true , address=185.167.204.206:4059 , piece_id=BYFO252V4VQSTUR5ZQH5Z5GPZ5UWQUNYYM5KEMOQD7T4QTWXO3RQ, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.344358107 (271h43m48.941129137s) [noise=true , address=185.167.204.205:4003 , piece_id=3ZOJYKYGEULH6DP6X75AOYEE7GP4HTUDNGSXPAKP75UJIA3HD32A, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.344521316 (271h43m48.941292356s) [noise=true , address=185.167.204.205:4026 , piece_id=IINFYHULKK3NVITBYGTGGZSTGVS5IHD4Y3PYJ2ZI44U4GJHYKEVQ, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:51.710308347 (271h43m49.307079362s) [noise=true , address=81.171.14.70:4034 , piece_id=N4ZIP4NLHWNI4YC4VS27M2SVAAOGLBOTF73UQTXLY63RZRO6XOGA, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:52.042022074 (271h43m49.638793087s) [noise=true , address=185.167.204.206:4050 , piece_id=2HVVFROYA4GVKMDG7RA6JB5SVB752NDMS6ST2AH7A7C7OCOPTYOA, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:43:52.537232280 (271h43m50.13400331s) [noise=true , address=185.167.204.205:4004 , piece_id=MGJYB33RC72AHOUQN7YW4CXVJU6BKLOFOPB7KNZZTUZNWMQVWQOA, seg_pos={0 0}, upload=304355]: Done uploading piece. err: <nil> Mar 17 18:44:50.186022442 (271h44m47.782793472s) [noise=true , address=89.149.201.147:4013 , piece_id=RJIXWWABM5EZEHNYLKRN3PX5PHUSPLV5ADSIKJYJ7FW3D2I2MY6A, seg_pos={0 0}, upload=304355]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: read tcp 159.223.143.39:50266->89.149.201.147:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 18:44:50.186208101 (271h44m47.782979127s) [noise=true , address=89.149.201.147:4005 , piece_id=6VFUFLRQX4F4C42UUYYWVCA5KMVMEJ2TMPGQFYYRM5MAYUMN6IWA, seg_pos={0 0}, upload=304355]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 159.223.143.39:53612->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 18:44:50.186223052 (271h44m47.782994068s) [noise=true , address=89.149.201.147:4018 , piece_id=HSIK6ZJDTRPR7NDVUJBHRT3SUT4UGIXSGZIW5GGBWIPPYX2VI7NA, seg_pos={0 0}, upload=304355]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:35522->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 17 18:44:50.186250571 (271h44m47.783021592s) [seg_pos={0 0}, upload=304355]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Mar 17 18:44:50.291559895 (271h44m47.888330926s) [upload=304355]: Done uploading segments... Mar 17 18:44:50.291613215 (271h44m47.888384241s) [upload=304355]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 2.571552452s core uplink eu-select-1 eu1-placement-15

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)