Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 399.309778ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:26:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 399.35053ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 444.615273ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:26:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 444.705298ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 697.603025ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:25:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 697.628798ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 706.731506ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:24:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 706.756765ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 1.596361434s trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:23:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1.596391246s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 731.196762ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:23:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 731.304736ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 523.213414ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:22:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 523.274244ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 394.265534ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:21:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 394.295271ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 470.738575ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:21:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 470.768775ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 463.812128ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:20:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 463.841677ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 464.240074ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:20:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 464.273611ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 411.624597ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:19:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 411.650341ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 393.54945ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:18:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 393.645308ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 374.205708ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:17:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 374.257339ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 430.962926ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:17:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 430.995813ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 412.666227ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:16:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 412.808131ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 416.673333ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:15:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 416.78743ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 424.505438ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:14:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 424.534465ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 433.682497ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:13:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 433.714332ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 447.534846ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:12:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 447.570827ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 414.219883ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:12:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 414.245791ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 417.423106ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:11:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 417.472185ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 444.168667ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:10:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 444.198033ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 455.537717ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:10:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 455.565184ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 401.1385ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:09:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 401.166254ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 455.795907ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:08:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 455.820466ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 477.780492ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:07:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 477.814859ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 415.261114ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:07:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 415.287083ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 494.04289ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:06:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 494.121004ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 444.282114ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:06:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 444.314182ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 484.837189ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:05:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 484.865511ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 583.724987ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:04:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 583.753649ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 477.645068ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:03:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 477.676555ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 518.152744ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:02:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 518.188431ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 439.443491ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:02:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 439.476635ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 413.136165ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:01:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 413.160629ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 539.690034ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:00:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 539.715652ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 406.963919ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:59:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 406.990693ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 398.763548ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:59:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 398.79587ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 424.990062ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:58:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 425.018881ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 445.709762ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:57:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 445.739641ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 556.859621ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:56:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 556.891026ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 504.443881ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:55:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 504.467811ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 438.972822ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:55:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 439.014869ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 496.77388ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:53:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 496.804931ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 570.863553ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:53:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 570.968967ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 674.701942ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:52:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 674.731456ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 407.102137ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:50:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 407.124834ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 423.853277ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:49:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 423.877835ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 486.610006ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:49:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 486.633937ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)