Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 430.962926ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:17:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 430.995813ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 412.666227ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:16:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 412.808131ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 416.673333ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:15:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 416.78743ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 424.505438ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:14:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 424.534465ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 433.682497ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:13:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 433.714332ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 447.534846ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:12:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 447.570827ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 414.219883ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:12:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 414.245791ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 417.423106ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:11:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 417.472185ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 444.168667ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:10:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 444.198033ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 455.537717ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:10:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 455.565184ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 401.1385ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:09:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 401.166254ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 455.795907ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:08:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 455.820466ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 477.780492ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:07:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 477.814859ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 415.261114ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:07:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 415.287083ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 494.04289ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:06:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 494.121004ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 444.282114ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:06:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 444.314182ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 484.837189ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:05:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 484.865511ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 583.724987ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:04:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 583.753649ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 477.645068ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:03:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 477.676555ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 518.152744ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:02:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 518.188431ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 439.443491ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:02:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 439.476635ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 413.136165ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:01:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 413.160629ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 539.690034ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:00:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 539.715652ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 406.963919ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:59:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 406.990693ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 398.763548ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:59:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 398.79587ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 424.990062ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:58:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 425.018881ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 445.709762ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:57:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 445.739641ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 556.859621ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:56:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 556.891026ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 504.443881ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:55:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 504.467811ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 438.972822ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:55:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 439.014869ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 496.77388ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:53:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 496.804931ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 570.863553ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:53:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 570.968967ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 674.701942ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:52:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 674.731456ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 407.102137ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:50:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 407.124834ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 423.853277ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:49:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 423.877835ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 486.610006ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:49:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 486.633937ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 377.161546ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:48:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 377.189972ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 435.115204ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:47:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 435.143897ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 423.396521ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:46:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 423.475563ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 782.057783ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:45:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 782.094187ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 480.56654ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:45:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 480.624027ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 423.129583ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:44:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 423.154874ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 667.107731ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:43:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 667.128773ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.039941428s trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:43:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.039972155s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [X...........................................................] duration: 273.845592ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:38:05 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 273.870163ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.X..........................................................] duration: 276.47582ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:37:35 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 276.511671ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..X.........................................................] duration: 285.715276ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:36:51 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 285.751046ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...X........................................................] duration: 322.639839ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:36:15 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 322.677257ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [....X.......................................................] duration: 267.255298ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:35:40 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 267.280934ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.....X......................................................] duration: 279.812755ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:34:51 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 279.894136ms edge linksharing bgp ap1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)